The three libraries contain identical sets of functions, with names ending in
_8, _16, or _32, respectively (for example, pcre2_compile_8()). However,
by defining PCRE2_CODE_UNIT_WIDTH to be 8, 16, or 32, a program that uses just
one code unit width can be written using generic names such as
pcre2_compile(), and the documentation is written assuming that this is
the case.
-
-
In addition to the Perl-compatible matching function, PCRE2 contains an
alternative function that matches the same compiled patterns in a different
way. In certain circumstances, the alternative function has some advantages.
For a discussion of the two matching algorithms, see the
pcre2matching
page.
-
-
Details of exactly which Perl regular expression features are and are not
supported by PCRE2 are given in separate documents. See the
pcre2pattern
@@ -83,8 +83,8 @@ and
pages. There is a syntax summary in the
pcre2syntax
page.
-
-
Some features of PCRE2 can be included, excluded, or changed when the library
is built. The
pcre2_config()
@@ -95,10 +95,10 @@ page. Documentation about building PCRE2 for various operating systems can be
found in the
README
and
-NON-AUTOTOOLS_BUILD
+NON-AUTOTOOLS-BUILD
files in the source distribution.
-
-
The libraries contains a number of undocumented internal functions and data
tables that are used by more than one of the exported external functions, but
which are not intended for use by external callers. Their names all begin with
@@ -106,9 +106,9 @@ which are not intended for use by external callers. Their names all begin with
environments, it is possible to control which external symbols are exported
when a shared library is built, and in these cases the undocumented symbols are
not exported.
-
-
If you are using PCRE2 in a non-UTF application that permits users to supply
arbitrary patterns for compilation, you should be aware of a feature that
allows users to turn on UTF support from within a pattern. For example, an
@@ -118,34 +118,34 @@ patterns and subjects as strings of UTF-8 code units instead of individual
matched to be checked for UTF-8 validity. If the data string is very long, such
a check might use sufficiently many resources as to cause your application to
lose performance.
-
-
The use of Unicode properties for character types such as \d can also be
enabled from within the pattern, by specifying "(*UCP)". This feature can be
disallowed by setting the PCRE2_NEVER_UCP option.
-
-
If your application is one that supports UTF, be aware that validity checking
can take time. If the same data string is to be matched many times, you can use
the PCRE2_NO_UTF_CHECK option for the second and subsequent matches to avoid
running redundant checks.
-
-
The use of the \C escape sequence in a UTF-8 or UTF-16 pattern can lead to
problems, because it may leave the current matching point in the middle of a
multi-code-unit character. The PCRE2_NEVER_BACKSLASH_C option can be used by an
application to lock out the use of \C, causing a compile-time error if it is
encountered. It is also possible to build PCRE2 with the use of \C permanently
disabled.
-
-
Another way that performance can be hit is by running a pattern that has a very
large search tree against a string that will never match. Nested unlimited
repeats in a pattern are a common example. PCRE2 provides some protection
@@ -153,9 +153,9 @@ against this: see the pcre2_set_match_limit() function in the
pcre2api
page. There is a similar function called pcre2_set_depth_limit() that can
be used to restrict the amount of memory that is used.
-
-
The user documentation for PCRE2 comprises a number of different sections. In
the "man" format, each of these is a separate "man page". In the HTML format,
each is a separate page, linked from the index page. In the plain text format,
@@ -189,23 +189,23 @@ listing), and the short pages for individual functions, are concatenated in
In the "man" and HTML formats, there is also a short page for each C library
function, listing its arguments and results.
-
-
PCRE2 was written by Philip Hazel, of the University Computing Service,
Cambridge, England. Many others have also contributed.
-
-
To contact the maintainers, please use the GitHub issues tracker or PCRE2
mailing list, as described at the project page:
https://github.com/PCRE2Project/pcre2
-
-
This function makes a copy of the memory used for a compiled pattern, excluding
any memory used by the JIT compiler. Without a subsequent call to
pcre2_jit_compile(), the copy can be used only for non-JIT matching. The
pointer to the character tables is copied, not the tables themselves (see
pcre2_code_copy_with_tables()). The yield of the function is NULL if
code is NULL or if sufficient memory cannot be obtained.
-
-
This function makes a copy of the memory used for a compiled pattern, excluding
any memory used by the JIT compiler. Without a subsequent call to
pcre2_jit_compile(), the copy can be used only for non-JIT matching.
@@ -32,8 +32,8 @@ Unlike pcre2_code_copy(), a separate copy of the character tables is also
made, with the new code pointing to it. This memory will be automatically freed
when pcre2_code_free() is called. The yield of the function is NULL if
code is NULL or if sufficient memory cannot be obtained.
-
-
This function compiles a regular expression pattern into an internal form. Its
arguments are:
There is a complete description of the PCRE2 native API, with more detail on
each option, in the
pcre2api
diff --git a/doc/html/pcre2_compile_context_copy.html b/doc/html/pcre2_compile_context_copy.html
index 9e9884b..d413015 100644
--- a/doc/html/pcre2_compile_context_copy.html
+++ b/doc/html/pcre2_compile_context_copy.html
@@ -12,25 +12,25 @@ This page is part of the PCRE2 HTML documentation. It was generated
automatically from the original man page. If there is any nonsense in it,
please consult the man page, in case the conversion went wrong.
-
+
SYNOPSIS
-
-
This function makes a new copy of a compile context, using the memory
allocation function that was used for the original context. The result is NULL
if the memory cannot be obtained.
-
-
This function creates and initializes a new compile context. If its argument is
NULL, malloc() is used to get the necessary memory; otherwise the memory
allocation function within the general context is used. The result is NULL if
the memory could not be obtained.
-
-
This function frees the memory occupied by a compile context, using the memory
freeing function from the general context with which it was created, or
free() if that was not set. If the argument is NULL, the function returns
immediately without doing anything.
-
-
This function makes it possible for a client program to find out which optional
features are available in the version of the PCRE2 library it is using. The
arguments are as follows:
@@ -35,45 +35,46 @@ arguments are as follows:
If where is NULL, the function returns the amount of memory needed for
the requested information. When the information is a string, the value is in
code units; for other types of data it is in bytes.
-
-
This function is part of an experimental set of pattern conversion functions.
It makes a new copy of a convert context, using the memory allocation function
that was used for the original context. The result is NULL if the memory cannot
be obtained.
-
-
This function is part of an experimental set of pattern conversion functions.
It creates and initializes a new convert context. If its argument is
NULL, malloc() is used to get the necessary memory; otherwise the memory
allocation function within the general context is used. The result is NULL if
the memory could not be obtained.
-
-
This function is part of an experimental set of pattern conversion functions.
It frees the memory occupied by a convert context, using the memory
freeing function from the general context with which it was created, or
free() if that was not set. If the argument is NULL, the function returns
immediately without doing anything.
-
-
This function is part of an experimental set of pattern conversion functions.
It frees the memory occupied by a converted pattern that was obtained by
calling pcre2_pattern_convert() with arguments that caused it to place
the converted pattern into newly obtained heap memory. If the argument is NULL,
the function returns immediately without doing anything.
-
-
This function matches a compiled regular expression against a given subject
string, using an alternative matching algorithm that scans the subject string
just once (except when processing lookaround assertions). This function is
@@ -49,8 +49,8 @@ The size of output vector needed to contain all the results depends on the
number of simultaneous matches, not on the number of parentheses in the
pattern. Using pcre2_match_data_create_from_pattern() to create the match
data block is therefore not advisable when using this function.
-
-
A match context is needed only if you want to set up a callout function or
specify the heap limit or the match or the recursion depth limits. The
length and startoffset values are code units, not characters. The
diff --git a/doc/html/pcre2_general_context_copy.html b/doc/html/pcre2_general_context_copy.html
index 0018534..fb2c296 100644
--- a/doc/html/pcre2_general_context_copy.html
+++ b/doc/html/pcre2_general_context_copy.html
@@ -12,26 +12,26 @@ This page is part of the PCRE2 HTML documentation. It was generated
automatically from the original man page. If there is any nonsense in it,
please consult the man page, in case the conversion went wrong.
-
+
SYNOPSIS
-
-
This function makes a new copy of a general context, using the memory
allocation functions in the context, if set, to get the necessary memory.
Otherwise malloc() is used. The result is NULL if the memory cannot be
obtained.
-
-
This function creates and initializes a general context. The arguments define
custom memory management functions and a data value that is passed to them when
they are called. The private_malloc() function is used to get memory for
the context. If either of the first two arguments is NULL, the system memory
management function is used. The result is NULL if no memory could be obtained.
-
-
This function frees the memory occupied by a general context, using the memory
freeing function within the context, if set. If the argument is NULL, the
function returns immediately without doing anything.
-
-
This function provides a textual error message for each PCRE2 error code.
Compilation errors are positive numbers; UTF formatting errors and matching
errors are negative numbers. The arguments are:
@@ -39,8 +39,8 @@ trailing zero, or the negative error code PCRE2_ERROR_NOMEMORY if the buffer is
too small. In this case, the returned message is truncated (but still with a
trailing zero). If errorcode does not contain a recognized error code
number, the negative value PCRE2_ERROR_BADDATA is returned.
-
-
After a successful match, the name that is returned is the last one on the
matching path. After a failed match or a partial match, the last encountered
name is returned.
-
-
This function returns the size, in bytes, of the heapframes data block that is
owned by its argument.
-
-
This function returns the size, in bytes, of the match data block that is its
argument.
-
-
This function returns the number of pairs of offsets in the ovector that forms
part of the given match data block.
-
-
This function returns a pointer to the vector of offsets that forms part of the
given match data block. The number of pairs can be found by calling
pcre2_get_ovector_count().
-
-
This function requests JIT compilation, which, if the just-in-time compiler is
available, further processes a compiled pattern into machine code that executes
much faster than the pcre2_match() interpretive matching function. Full
details are given in the
pcre2jit
documentation.
-
-
Otherwise, the first argument must be a pointer that was returned by a
successful call to pcre2_compile(), and the second must contain one or
more of the following bits:
@@ -53,8 +53,8 @@ more of the following bits:
There is also an obsolete option called PCRE2_JIT_INVALID_UTF, which has been
superseded by the pcre2_compile() option PCRE2_MATCH_INVALID_UTF. The old
option is deprecated and may be removed in the future.
-
-
The yield of the function when called with any of the three options above is 0
for success, or a negative error code otherwise. In particular,
PCRE2_ERROR_JIT_BADOPTION is returned if JIT is not supported or if an unknown
@@ -62,8 +62,8 @@ bit is set in options. The function can also return PCRE2_ERROR_NOMEMORY
if JIT is unable to allocate executable memory for the compiler, even if it was
because of a system security restriction. In a few cases, the function may
return with PCRE2_ERROR_JIT_UNSUPPORTED for unsupported features.
-
-
This function frees unused JIT executable memory. The argument is a general
context, for custom memory management, or NULL for standard memory management.
JIT memory allocation retains some memory in order to improve future JIT
compilation speed. In low memory conditions,
pcre2_jit_free_unused_memory() can be used to cause this memory to be
freed.
-
-
This function matches a compiled regular expression that has been successfully
processed by the JIT compiler against a given subject string, using a matching
algorithm that is similar to Perl's. It is a "fast path" interface to JIT, and
it bypasses some of the sanity checks that pcre2_match() applies.
-
-
In UTF mode, the subject string is not checked for UTF validity. Unless
PCRE2_MATCH_INVALID_UTF was set when the pattern was compiled, passing an
invalid UTF string results in undefined behaviour. Your program may crash or
loop or give wrong results. In the absence of PCRE2_MATCH_INVALID_UTF you
should only call pcre2_jit_match() in UTF mode if you are sure the
subject is valid.
-
-
The supported options are PCRE2_NOTBOL, PCRE2_NOTEOL, PCRE2_NOTEMPTY,
PCRE2_NOTEMPTY_ATSTART, PCRE2_PARTIAL_HARD, and PCRE2_PARTIAL_SOFT. Unsupported
options are ignored.
-
-
This function provides control over the memory used by JIT as a run-time stack
when pcre2_match() or pcre2_jit_match() is called with a pattern
that has been successfully processed by the JIT compiler. The information that
@@ -35,36 +35,36 @@ passed to a matching function. The arguments of this function are:
mcontext a pointer to a match context
callback a callback function
callback_data a JIT stack or a value to be passed to the callback
-
-
-
You may safely use the same JIT stack for multiple patterns, as long as they
are all matched in the same thread. In a multithread application, each thread
must use its own JIT stack. For more details, see the
pcre2jit
page.
-
-
This function is used to create a stack for use by the code compiled by the JIT
compiler. The first two arguments are a starting size for the stack, and a
maximum size to which it is allowed to grow. The final argument is a general
@@ -38,8 +38,8 @@ pattern. If the stack couldn't be allocated or the values passed were not
reasonable, NULL will be returned. For more details, see the
pcre2jit
page.
-
-
This function builds a set of character tables for character code points that
are less than 256. These can be passed to pcre2_compile() in a compile
context in order to override the internal, built-in tables (which were either
defaulted or made by pcre2_maketables() when PCRE2 was compiled). See the
pcre2_set_character_tables()
page. You might want to do this if you are using a non-standard locale.
-
-
This function discards a set of character tables that were created by a call
to
pcre2_maketables().
-
-
This function matches a compiled regular expression against a given subject
string, using a matching algorithm that is similar to Perl's. It returns
offsets to what it has matched and to captured substrings via the
diff --git a/doc/html/pcre2_match_context_copy.html b/doc/html/pcre2_match_context_copy.html
index 4a719d6..068f7f9 100644
--- a/doc/html/pcre2_match_context_copy.html
+++ b/doc/html/pcre2_match_context_copy.html
@@ -12,25 +12,25 @@ This page is part of the PCRE2 HTML documentation. It was generated
automatically from the original man page. If there is any nonsense in it,
please consult the man page, in case the conversion went wrong.
-
+
SYNOPSIS
-
-
This function makes a new copy of a match context, using the memory
allocation function that was used for the original context. The result is NULL
if the memory cannot be obtained.
-
-
This function creates and initializes a new match context. If its argument is
NULL, malloc() is used to get the necessary memory; otherwise the memory
allocation function within the general context is used. The result is NULL if
the memory could not be obtained.
-
-
This function frees the memory occupied by a match context, using the memory
freeing function from the general context with which it was created, or
free() if that was not set. If the argument is NULL, the function returns
immediately without doing anything.
-
-
This function creates a new match data block, which is used for holding the
result of a match. The first argument specifies the number of pairs of offsets
that are required. These form the "output vector" (ovector) within the match
@@ -33,13 +33,13 @@ data block, and are used to identify the matched string and any captured
substrings when matching with pcre2_match(), or a number of different
matches at the same point when used with pcre2_dfa_match(). There is
always one pair of offsets; if ovecsize is zero, it is treated as one.
-
-
The second argument points to a general context, for custom memory management,
or is NULL for system memory management. The result of the function is NULL if
the memory for the block could not be obtained.
-
-
This function creates a new match data block for holding the result of a match.
-The first argument points to a compiled pattern. The number of capturing
-parentheses within the pattern is used to compute the number of pairs of
-offsets that are required in the match data block. These form the "output
-vector" (ovector) within the match data block, and are used to identify the
-matched string and any captured substrings when matching with
-pcre2_match(). If you are using pcre2_dfa_match(), which uses the
-output vector in a different way, you should use pcre2_match_data_create()
-instead of this function.
-
-
+If the first argument is NULL, this function returns NULL, otherwise the first
+argument points to a compiled pattern. The number of capturing parentheses
+within the pattern is used to compute the number of pairs of offsets that are
+required in the match data block. These form the "output vector" (ovector)
+within the match data block, and are used to identify the matched string and
+any captured substrings when matching with pcre2_match(). If you are
+using pcre2_dfa_match(), which uses the output vector in a different way,
+you should use pcre2_match_data_create() instead of this function.
+
+
The second argument points to a general context, for custom memory management,
-or is NULL to use the same memory allocator as was used for the compiled
+or is NULL to use the same memory allocator that was used for the compiled
pattern. The result of the function is NULL if the memory for the block could
-not be obtained.
-
-
If the PCRE2_COPY_MATCHED_SUBJECT was used for a successful match using this
match data block, the copy of the subject that was referenced within the block
is also freed.
-
-
+This page is part of the PCRE2 HTML documentation. It was generated
+automatically from the original man page. If there is any nonsense in it,
+please consult the man page, in case the conversion went wrong.
+
+
+It returns 0 ("false") if there is no need to make any further match attempts,
+or 1 ("true") if another match should be attempted.
+
+
This function is part of an experimental set of pattern conversion functions.
It converts a foreign pattern (for example, a glob) into a PCRE2 regular
expression pattern. Its arguments are:
@@ -46,8 +46,8 @@ present (or else malloc()), and the field pointed to by buffer is
updated. If buffer points to a non-NULL field, that must point to a
buffer whose size is in the variable pointed to by blength. This value is
updated.
-
-
This function returns information about a compiled pattern. Its arguments are:
This function decodes a serialized set of compiled patterns back into a list of
individual patterns. This is possible only on a host that is running the same
version of PCRE2, with the same code unit width, and the host must also have
@@ -53,8 +53,8 @@ the following negative error codes:
PCRE2_ERROR_BADMAGIC may mean that the data is corrupt, or that it was compiled
on a system with different endianness.
-
-
This function encodes a list of compiled patterns into a byte stream that can
be saved on disc or elsewhere. Note that this is not an abstract format like
Java or .NET. Conversion of the byte stream back into usable compiled patterns
@@ -54,8 +54,8 @@ serialized patterns, or one of the following negative error codes:
PCRE2_ERROR_BADMAGIC means either that a pattern's code has been corrupted, or
that a slot in the vector does not point to a compiled pattern.
-
-
This function sets the convention for processing \R within a compile context.
The second argument must be one of PCRE2_BSR_ANYCRLF or PCRE2_BSR_UNICODE. The
result is zero for success or PCRE2_ERROR_BADDATA if the second argument is
invalid.
-
-
This function sets the callout fields in a match context (the first argument).
The second argument specifies a callout function, and the third argument is an
opaque data item that is passed to it. The result of this function is always
zero.
-
-
This function sets a pointer to custom character tables within a compile
context. The second argument must point to a set of PCRE2 character tables or
be NULL to request the default tables. The result is always zero. Character
@@ -33,8 +33,8 @@ tables can be created by calling pcre2_maketables() or by running the
pcre2_dftables maintenance command in binary mode (see the
pcre2build
documentation).
-
-
This function defines, within a compile context, a function that is called
whenever pcre2_compile() starts to compile a parenthesized part of a
pattern. The first argument to the function gives the current depth of
@@ -34,8 +34,8 @@ function is set up. The callout function should return zero if all is well, or
non-zero to force an error. This feature is provided so that applications can
check the available system stack space, in order to avoid running out. The
result of pcre2_set_compile_recursion_guard() is always zero.
-
-
This function sets the backtracking depth limit field in a match context. The
result is always zero.
-
-
This function is part of an experimental set of pattern conversion functions.
It sets the escape character that is used when converting globs. The second
argument must either be zero (meaning there is no escape character) or a
@@ -33,8 +33,8 @@ punctuation character whose code point is less than 256. The default is grave
accent if running under Windows, otherwise backslash. The result of the
function is zero for success or PCRE2_ERROR_BADDATA if the second argument is
invalid.
-
-
This function is part of an experimental set of pattern conversion functions.
It sets the component separator character that is used when converting globs.
The second argument must be one of the characters forward slash, backslash, or
dot. The default is backslash when running under Windows, otherwise forward
slash. The result of the function is zero for success or PCRE2_ERROR_BADDATA if
the second argument is invalid.
-
-
This function sets the backtracking heap limit field in a match context. The
result is always zero.
-
-
This function sets the match limit field in a match context. The result is
always zero.
-
-
This function sets, in a compile context, the maximum size (in bytes) for the
memory needed to hold the compiled version of a pattern that is using this
context. The result is always zero. If a pattern that is passed to
pcre2_compile() referencing this context needs more memory, an error is
generated. The default is the largest number that a PCRE2_SIZE variable can
hold, which is effectively unlimited.
-
-
This function sets, in a compile context, the maximum text length (in code
units) of the pattern that can be compiled. The result is always zero. If a
longer pattern is passed to pcre2_compile() there is an immediate error
return. The default is effectively unlimited, being the largest value a
PCRE2_SIZE variable can hold.
-
-
This sets a maximum length for the number of characters matched by a
variable-length lookbehind assertion. The default is set when PCRE2 is built,
with the ultimate default being 255, the same as Perl. Lookbehind assertions
without a bounding length are not supported. The result is always zero.
-
-
This function sets the newline convention within a compile context. This
specifies which character(s) are recognized as newlines when compiling and
matching patterns. The second argument must be one of:
@@ -39,8 +39,8 @@ matching patterns. The second argument must be one of:
The result is zero for success or PCRE2_ERROR_BADDATA if the second argument is
invalid.
-
-
This function sets the offset limit field in a match context. The result is
always zero.
-
-REVISION
-
SYNOPSIS
-
+
+
pcre2test [options] [input file [output file]]
@@ -49,24 +49,24 @@ documentation. For details of the PCRE2 library function calls and their
options, see the
pcre2api
documentation.
-
-
+
+
The input for pcre2test is a sequence of regular expression patterns and
subject strings to be matched. There are also command lines for setting
defaults and controlling some special actions. The output shows the result of
each match attempt. Modifiers on external or internal command lines, the
patterns, and the subject lines specify PCRE2 function options, control how the
subject is processed, and what output is produced.
-
-
+
+
There are many obscure modifiers, some of which are specifically designed for
use in conjunction with the test script and data files that are distributed as
part of PCRE2. All the modifiers are documented here, some without much
justification, but many of them are unlikely to be of use except when testing
the libraries.
-
-
PCRE2's 8-BIT, 16-BIT AND 32-BIT LIBRARIES
-
+
+
+
Different versions of the PCRE2 library can be built to support character
strings that are encoded in 8-bit, 16-bit, or 32-bit code units. One, two, or
all three of these libraries may be simultaneously installed. The
@@ -75,21 +75,21 @@ input and output are always in 8-bit format. When testing the 16-bit or 32-bit
libraries, patterns and subject strings are converted to 16-bit or 32-bit
format before being passed to the library functions. Results are converted back
to 8-bit code units for output.
-
-
+
+
In the rest of this document, the names of library functions and structures
are given in generic form, for example, pcre2_compile(). The actual
names used in the libraries have a suffix _8, _16, or _32, as appropriate.
-
-
INPUT ENCODING
-
+
+
+
Input to pcre2test is processed line by line, either by calling the C
library's fgets() function, or via the libreadline or libedit
library. In some Windows environments character 26 (hex 1A) causes an immediate
end of file, and no further data is read, so this character should be avoided
unless you really want that action.
-
-
+
+
The input is processed using C's string functions, so must not contain binary
zeros, even though in Unix-like environments, fgets() treats any bytes
other than newline as data characters. An error is generated if a binary zero
@@ -98,11 +98,11 @@ which makes it possible to include any data value in strings that are passed to
the library for matching. For patterns, there is a facility for specifying some
or all of the 8-bit input characters as hexadecimal pairs, which makes it
possible to include binary zeros.
-
-
+
+
Input for the 16-bit and 32-bit libraries
-
-
+
+
When testing the 16-bit or 32-bit libraries, there is a need to be able to
generate character code points greater than 255 in the strings that are passed
to the library. For subject lines and some patterns, backslash escapes can be
@@ -110,8 +110,8 @@ used. In addition, when the utf modifier (see
"Setting compilation options"
below) is set, the pattern and any following subject lines are interpreted as
UTF-8 strings and translated to UTF-16 or UTF-32 as appropriate.
-
-
+
+
For non-UTF testing of wide characters, the utf8_input modifier can be
used. This is mutually exclusive with utf, and is allowed only in 16-bit
or 32-bit mode. It causes the pattern and following subject lines to be treated
@@ -119,67 +119,63 @@ as UTF-8 according to the original definition (RFC 2279), which allows for
character values up to 0x7fffffff. Each character is placed in one 16-bit or
32-bit code unit (in the 16-bit case, values greater than 0xffff cause an error
to occur).
-
-
+
+
UTF-8 (in its original definition) is not capable of encoding values greater
than 0x7fffffff, but such values can be handled by the 32-bit library. When
testing this library in non-UTF mode with utf8_input set, if any
character is preceded by the byte 0xff (which is an invalid byte in UTF-8)
0x80000000 is added to the character's value. For subject strings, using an
escape sequence is preferable.
-
-
COMMAND LINE OPTIONS
-
+
+
+
-8
If the 8-bit library has been built, this option causes it to be used (this is
the default). If the 8-bit library has not been built, this option causes an
error.
-
-
+
+
-16
If the 16-bit library has been built, this option causes it to be used. If the
8-bit library has not been built, this is the default. If the 16-bit library
has not been built, this option causes an error.
-
-
+
+
-32
If the 32-bit library has been built, this option causes it to be used. If no
other library has been built, this is the default. If the 32-bit library has
not been built, this option causes an error.
-
-
+
+
-ac
Behave as if each pattern has the auto_callout modifier, that is, insert
automatic callouts into every pattern that is compiled.
-
-
+
+
-AC
As for -ac, but in addition behave as if each subject line has the
callout_extra modifier, that is, show additional information from
callouts.
-
-
+
+
-b
Behave as if each pattern has the fullbincode modifier; the full
internal binary form of the pattern is output after compilation.
-
-
+
+
-C
Output the version number of the PCRE2 library, and all available information
about the optional features that are included, and then exit with zero exit
code. All other options are ignored. If both -C and -LM are present, whichever
is first is recognized.
-
-
+
+
-C option
Output information about a specific build-time option, then exit. This
functionality is intended for use in scripts such as RunTest. The
following options output the value and set the exit code as indicated:
- ebcdic-nl the code for LF (= NL) in an EBCDIC environment:
- either 0x15 or 0x25
- 0 if used in an ASCII/Unicode environment
- exit code is always 0
linksize the configured internal link size (2, 3, or 4)
exit code is set to the link size
newline the default newline setting:
@@ -194,6 +190,10 @@ to the same value:
backslash-C \C is supported (not locked out)
ebcdic compiled for an EBCDIC environment
+ ebcdic-io if PCRE2 is compiled for EBCDIC, whether pcre2test's input and
+ output is EBCDIC or ASCII
+ ebcdic-nl25 if PCRE2 is compiled for EBCDIC, whether NL (= LF) is 0x25
+ (otherwise it is 0x15, the default)
jit just-in-time support is available
pcre2-16 the 16-bit library was built
pcre2-32 the 32-bit library was built
@@ -211,89 +211,106 @@ information. It returns one of the following values:
3 Unexpected return from test call to pcre2_jit_compile()
If an unknown option is given, an error message is output; the exit code is 0.
-
-
+
+
+--colo[u]r[=<always,auto,never>]
+By default, the output is coloured if the output file is a terminal (auto).
+Force or suppress output of ANSI colour escapes with always and never
+respectively.
+
+
-d
Behave as if each pattern has the debug modifier; the internal
form and information about the compiled pattern is output after compilation;
-d is equivalent to -b -i.
-
-
+
+
-dfa
Behave as if each subject line has the dfa modifier; matching is done
using the pcre2_dfa_match() function instead of the default
pcre2_match().
-
-
+
+
+-E
+Run in "preprocess only" mode (similar to "gcc -E"). The "#if ... #endif"
+commands are processed, and all other lines are printed verbatim.
+
+
-error number[,number,...]
Call pcre2_get_error_message() for each of the error numbers in the
comma-separated list, display the resulting messages on the standard output,
then exit with zero exit code. The numbers may be positive or negative. This is
a convenience facility for PCRE2 maintainers.
-
-
+
+
-help
Output a brief summary these options and then exit.
-
-
+
+
-i
Behave as if each pattern has the info modifier; information about the
compiled pattern is given after compilation.
-
-
+
+
-jit
Behave as if each pattern line has the jit modifier; after successful
compilation, each pattern is passed to the just-in-time compiler, if available.
-
-
+
+
-jitfast
Behave as if each pattern line has the jitfast modifier; after
successful compilation, each pattern is passed to the just-in-time compiler, if
available, and each subject line is passed directly to the JIT matcher via its
"fast path".
-
-
+
+
-jitverify
Behave as if each pattern line has the jitverify modifier; after
successful compilation, each pattern is passed to the just-in-time compiler, if
available, and the use of JIT for matching is verified.
-
-
+
+
-LM
List modifiers: write a list of available pattern and subject modifiers to the
standard output, then exit with zero exit code. All other options are ignored.
If both -C and any -Lx options are present, whichever is first is recognized.
-
-
+
+
-LP
List properties: write a list of recognized Unicode properties to the standard
output, then exit with zero exit code. All other options are ignored. If both
-C and any -Lx options are present, whichever is first is recognized.
-
-
+
+
-LS
List scripts: write a list of recognized Unicode script names to the standard
output, then exit with zero exit code. All other options are ignored. If both
-C and any -Lx options are present, whichever is first is recognized.
-
-
+
+
+-malloc
+Exercise malloc() failures, by first counting the number of calls made to malloc
+during pattern compilation and matching, then re-running the compilation and
+matching that many times, exercising a failure of each malloc() call.
+
+
-pattern modifier-list
Behave as if each pattern line contains the given modifiers.
-
-
+
+
-q
Do not output the version number of pcre2test at the start of execution.
-
-
+
+
-S size
On Unix-like systems, set the size of the run-time stack to size
mebibytes (units of 1024*1024 bytes).
-
-
+
+
-subject modifier-list
Behave as if each subject line contains the given modifiers.
-
-
+
+
-t
Run each compile and match many times with a timer, and output the resulting
times per compile or match. When JIT is used, separate times are given for the
@@ -301,37 +318,42 @@ initial compile and the JIT compile. You can control the number of iterations
that are used for timing by following -t with a number (as a separate
item on the command line). For example, "-t 1000" iterates 1000 times. The
default is to iterate 500,000 times.
-
-
+
+
-tm
This is like -t except that it times only the matching phase, not the
compile phase.
-
-
+
+
-T -TM
These behave like -t and -tm, but in addition, at the end of a run,
the total times for all compiles and matches are output.
-
-
+
+
+-unittest
+Run a fixed set of additional tests of the PCRE2 API which are not driven by
+the test input files, and then exit.
+
+
-version
Output the PCRE2 version number and then exit.
-
-
DESCRIPTION
-
+
+
+
If pcre2test is given two filename arguments, it reads from the first and
writes to the second. If the first name is "-", input is taken from the
standard input. If pcre2test is given only one argument, it reads from
that file and writes to stdout. Otherwise, it reads from stdin and writes to
stdout.
-
-
+
+
When pcre2test is built, a configuration option can specify that it
should be linked with the libreadline or libedit library. When this
is done, if the input is from a terminal, it is read using the readline()
function. This provides line-editing and history facilities. The output from
the -help option states whether or not readline() will be used.
-
-
+
+
The program handles any number of tests, each of which consists of a set of
input lines. Each set starts with a regular expression pattern, followed by any
number of subject lines to be matched against that pattern. In between sets of
@@ -340,14 +362,14 @@ some restrictions, can also be processed by the perltest.sh script that
is distributed with PCRE2 as a means of checking that the behaviour of PCRE2
and Perl is the same. For a specification of perltest.sh, see the
comments near its beginning. See also the #perltest command below.
-
-
+
+
When the input is a terminal, pcre2test prompts for each line of input,
using "re>" to prompt for regular expression patterns, and "data>" to prompt
for subject lines. Command lines starting with # can be entered only in
response to the "re>" prompt.
-
-
+
+
Each subject line is matched separately and independently. If you want to do
multi-line matches, you have to use the \n escape sequence (or \r or \r\n,
etc., depending on the newline setting) in a single line of input to encode the
@@ -355,14 +377,14 @@ newline sequences. There is no limit on the length of subject lines; the input
buffer is automatically extended if it is too small. There are replication
features that makes it possible to generate long repetitive pattern or subject
lines without having to supply them explicitly.
-
-
+
+
An empty line or the end of the file signals the end of the subject lines for a
test, at which point a new pattern or command line is expected if there is
still input to be read.
-
-
COMMAND LINES
-
+
+
+
In between sets of test data, a line that begins with # is interpreted as a
command line. If the first character is followed by white space or an
exclamation mark, the line is treated as a comment, and ignored. Otherwise, the
@@ -376,8 +398,8 @@ the use of (*UTF) and (*UCP) at the start of patterns. This command also forces
an error if a subsequent pattern contains any occurrences of \P, \p, or \X,
which are still supported when PCRE2_UTF is not set, but which require Unicode
property support to be included in the library.
-
-
+
+
This is a trigger guard that is used in test files to ensure that UTF or
Unicode property tests are not accidentally added to files that are used when
Unicode support is not included in the library. Setting PCRE2_NEVER_UTF and
@@ -407,8 +429,8 @@ pattern is compiled. The standard test files contain tests of various newline
conventions, but the majority of the tests expect a single linefeed to be
recognized as a newline by default. Without special action the tests would fail
when PCRE2 is compiled with either CR or CRLF as the default newline.
-
-
+
+
The #newline_default command specifies a list of newline types that are
acceptable as the default. The types must be one of CR, LF, CRLF, ANYCRLF,
ANY, or NUL (in upper or lower case), for example:
@@ -421,8 +443,8 @@ first newline convention in the list (LF in the above example) is added to any
pattern that does not already have a newline modifier. If the newline
list is empty, the feature is turned off. This command is present in a number
of the standard test input files.
-
-
+
+
When the POSIX API is being tested there is no way to override the default
newline convention, though it is possible to set the newline convention from
within the pattern. A warning is given if the posix or posix_nosub
@@ -440,8 +462,8 @@ This line is used in test files that can also be processed by perltest.sh
to confirm that Perl gives the same results as PCRE2. Subsequent tests are
checked for the use of pcre2test features that are incompatible with the
perltest.sh script.
-
-
+
+
Patterns must use '/' as their delimiter, and only certain modifiers are
supported. Comment lines, #pattern commands, and #subject commands that set or
unset "mark" are recognized and acted on. The #perltest, #forbid_utf, and
@@ -468,19 +490,30 @@ in the section entitled "Saving and restoring compiled patterns"
This command sets a default modifier list that applies to all subsequent
subject lines. Modifiers on a subject line can change these settings.
-
-
MODIFIER SYNTAX
-
+
+ #if CONDITION
+ ...
+ #endif
+
+If CONDITION is true, then the command is printed, and its contents are
+processed as normal, including printing the commandlines to the output. If
+CONDITION is false, then all lines between the "#if" and "#endif" are skipped
+and not printed. The CONDITION can be any of the conditions which are tested by
+the "-C" commandline option and which set pcre2test's exit code to a boolean
+value. The CONDITION may also be preceded by "!".
+
+
+
Modifier lists are used with both pattern and subject lines. Items in a list
-are separated by commas followed by optional white space. Trailing whitespace
+are separated by commas followed by optional white space. Trailing white space
in a modifier list is ignored. Some modifiers may be given for both patterns
and subject lines, whereas others are valid only for one or the other. Each
modifier has a long name, for example "anchored", and some of them must be
followed by an equals sign and a value, for example, "offset=12". Values cannot
contain comma characters, but may contain spaces. Modifiers that do not take
values may be preceded by a minus sign to turn off a previous setting.
-
-
+
+
A few of the more common modifiers can also be specified as single letters, for
example "i" for "caseless". In documentation, following the Perl convention,
these are written with a slash ("the /i modifier") for clarity. Abbreviated
@@ -492,9 +525,9 @@ sequence of these abbreviations. For example:
This is a pattern line whose modifier list starts with two one-letter modifiers
(/i and /g). The lower-case abbreviated modifiers are the same as used in Perl.
-
-
PATTERN SYNTAX
-
+
+
+
A pattern line must start with one of the following characters (common symbols,
excluding pattern meta-characters):
@@ -524,12 +557,12 @@ backslash, because
is interpreted as the first line of a pattern that starts with "abc/", causing
pcre2test to read the next line as a continuation of the regular expression.
-
-
+
+
A pattern can be followed by a modifier list (details below).
-
-
SUBJECT LINE SYNTAX
-
+
+
+
Before each subject line is passed to pcre2_match(),
pcre2_dfa_match(), or pcre2_jit_match(), leading and trailing white
space is removed, and the line is scanned for backslash escapes, unless the
@@ -558,8 +591,8 @@ modifier on the pattern. It is always recognized. There may be any number of
hexadecimal digits inside the braces; invalid values provoke error messages
but when using \N{U+hh...} with some invalid unicode characters they will
be accepted with a warning instead.
-
-
+
+
Note that even in UTF-8 mode, \xhh (and depending of how large, \ddd)
describe one byte rather than one character; this makes it possible to
construct invalid UTF-8 sequences for testing purposes. On the other hand,
@@ -568,18 +601,18 @@ more than one byte if the value is greater than 127. To avoid the ambiguity
it is preferred to use \N{U+hh...} when describing characters. When testing
the 8-bit library not in UTF-8 mode, \x{hh} generates one byte for values
that could fit on it, and causes an error for greater values.
-
-
+
+
When testing the 16-bit library, not in UTF-16 mode, all 4-digit \x{hhhh}
values are accepted. This makes it possible to construct invalid UTF-16
sequences for testing purposes.
-
-
+
+
When testing the 32-bit library, not in UTF-32 mode, all 4 to 8-digit \x{...}
values are accepted. This makes it possible to construct invalid UTF-32
sequences for testing purposes.
-
-
+
+
There is a special backslash sequence that specifies replication of one or more
characters:
@@ -592,14 +625,14 @@ part of the file. For example:
is converted to "abcabcabcabc". This feature does not support nesting. To
include a closing square bracket in the characters, code it as \x5D.
-
-
+
+
A backslash followed by an equals sign marks the end of the subject string and
the start of a modifier list. For example:
abc\=notbol,notempty
-If the subject string is empty and \= is followed by whitespace, the line is
+If the subject string is empty and \= is followed by white space, the line is
treated as a comment line, and is not used for matching. For example:
\= This is a comment.
@@ -610,24 +643,24 @@ character. A backslash followed by anything else causes an error. However, if
the very last character in the line is a backslash (and there is no modifier
list), it is ignored. This gives a way of passing an empty line as data, since
a real empty line terminates the data input.
-
-
+
+
If the subject_literal modifier is set for a pattern, all subject lines
that follow are treated as literals, with no special treatment of backslashes.
No replication is possible, and any subject modifiers must be set as defaults
by a #subject command.
-
-
PATTERN MODIFIERS
-
+
+
+
There are several types of modifier that can appear in pattern lines. Except
where noted below, they may also be used in #pattern commands. A
pattern's modifier list can add to or override default modifiers that were set
by a previous #pattern command.
-
-
+
+
Setting compilation options
-
-
+
+
The following modifiers set options for pcre2_compile(). Most of them set
bits in the options argument of that function, but those whose names start with
PCRE2_EXTRA are additional options that are set in the compile context.
@@ -711,11 +744,11 @@ calling pcre2_set_optimize() before invoking the regex compiler.
See the
pcre2_set_optimize
documentation for details on these optimizations.
-
-
+
+
Setting compilation controls
-
-
+
+
The following modifiers affect the compilation process or request information
about the pattern. There are single-letter abbreviations for some that are
heavily used in the test files.
@@ -758,41 +791,41 @@ heavily used in the test files.
utf8_input treat input as UTF-8
The effects of these modifiers are described in the following sections.
-
-
+
+
Newline and \R handling
-
-
+
+
The bsr modifier specifies what \R in a pattern should match. If it is
set to "anycrlf", \R matches CR, LF, or CRLF only. If it is set to "unicode",
\R matches any Unicode newline sequence. The default can be specified when
PCRE2 is built; if it is not, the default is set to Unicode.
-
-
+
+
The newline modifier specifies which characters are to be interpreted as
newlines, both in the pattern and in subject lines. The type must be one of CR,
LF, CRLF, ANYCRLF, ANY, or NUL (in upper or lower case).
-
-
+
+
Information about a pattern
-
-
+
+
The debug modifier is a shorthand for info,fullbincode, requesting
all available information.
-
-
+
+
The bincode modifier causes a representation of the compiled code to be
output after compilation. This information does not contain length and offset
values, which ensures that the same output is generated for different internal
link sizes and different code unit widths. By using bincode, the same
regression tests can be used in different environments.
-
-
+
+
The fullbincode modifier, by contrast, does include length and
offset values. This is used in a few special tests that run only for specific
code unit widths and link sizes, and is also useful for one-off tests.
-
-
+
+
The info modifier requests information about the compiled pattern
(whether it is anchored, has a fixed first character, and so on). The
information is obtained from the pcre2_pattern_info() function. Here are
@@ -823,42 +856,42 @@ not necessarily the last character. These lines are omitted if no starting or
ending code units are recorded. The subject length line is omitted when
no_start_optimize is set because the minimum length is not calculated
when it can never be used.
-
-
+
+
The framesize modifier shows the size, in bytes, of each storage frame
used by pcre2_match() for handling backtracking. The size depends on the
number of capturing parentheses in the pattern. A vector of these frames is
used at matching time; its overall size is shown when the heaframes_size
subject modifier is set.
-
-
+
+
The callout_info modifier requests information about all the callouts in
the pattern. A list of them is output at the end of any other information that
is requested. For each callout, either its number or string is given, followed
by the item that follows it in the pattern.
-
-
+
+
Passing a NULL context
-
-
+
+
Normally, pcre2test passes a context block to pcre2_compile(). If
the null_context modifier is set, however, NULL is passed. This is for
testing that pcre2_compile() behaves correctly in this case (it uses
default values).
-
-
+
+
Passing a NULL pattern
-
-
+
+
The null_pattern modifier is for testing the behaviour of
pcre2_compile() when the pattern argument is NULL. The length value
passed is the default PCRE2_ZERO_TERMINATED unless use_length is set.
Any length other than zero causes an error.
-
-
+
+
Specifying pattern characters in hexadecimal
-
-
+
+
The hex modifier specifies that the characters of the pattern, except for
substrings enclosed in single or double quotes, are to be interpreted as pairs
of hexadecimal digits. This feature is provided as a way of creating patterns
@@ -876,37 +909,37 @@ nine characters, only two of which are specified in hexadecimal:
Either single or double quotes may be used. There is no way of including
the delimiter within a substring. The hex and expand modifiers are
mutually exclusive.
-
-
+
+
Specifying the pattern's length
-
-
+
+
By default, patterns are passed to the compiling functions as zero-terminated
strings but can be passed by length instead of being zero-terminated. The
use_length modifier causes this to happen. Using a length happens
automatically (whether or not use_length is set) when hex is set,
because patterns specified in hexadecimal may contain binary zeros.
-
-
+
+
If hex or use_length is used with the POSIX wrapper API (see
"Using the POSIX wrapper API"
below), the REG_PEND extension is used to pass the pattern's length.
-
-
+
+
Specifying a maximum for variable lookbehinds
-
-
+
+
Variable lookbehind assertions are supported only if, for each one, there is a
maximum length (in characters) that it can match. There is a limit on this,
whose default can be set at build time, with an ultimate default of 255. The
max_varlookbehind modifier uses the pcre2_set_max_varlookbehind()
function to change the limit. Lookbehinds whose branches each match a fixed
length are limited to 65535 characters per branch.
-
-
+
+
Specifying wide characters in 16-bit and 32-bit modes
-
-
+
+
In 16-bit and 32-bit modes, all input is automatically treated as UTF-8 and
translated to UTF-16 or UTF-32 when the utf modifier is set. For testing
the 16-bit and 32-bit libraries in non-UTF mode, the utf8_input modifier
@@ -915,11 +948,11 @@ interpreted as UTF-8 as a means of specifying wide characters. More details are
given in
"Input encoding"
above.
-
-
+
+
Generating long repetitive patterns
-
-
+
+
Some tests use long patterns that are very repetitive. Instead of creating a
very long input line for such a pattern, you can use a special repetition
feature, similar to the one described for subject lines above. If the
@@ -934,21 +967,21 @@ cannot be nested. An initial "\[" sequence is recognized only if "]{" followed
by decimal digits and "}" is found later in the pattern. If not, the characters
remain in the pattern unaltered. The expand and hex modifiers are
mutually exclusive.
-
-
+
+
If part of an expanded pattern looks like an expansion, but is really part of
the actual pattern, unwanted expansion can be avoided by giving two values in
the quantifier. For example, \[AB]{6000,6000} is not recognized as an
expansion item.
-
-
+
+
If the info modifier is set on an expanded pattern, the result of the
expansion is included in the information that is output.
-
-
+
+
JIT compilation
-
-
+
+
Just-in-time (JIT) compiling is a heavyweight optimization that can greatly
speed up pattern matching. See the
pcre2jit
@@ -960,8 +993,8 @@ different code is generated for the different cases. See the partial
modifier in "Subject Modifiers"
below
for details of how these options are specified for each match attempt.
-
-
+
+
JIT compilation is requested by the jit pattern modifier, which may
optionally be followed by an equals sign and a number in the range 0 to 7.
The three bits that make up the number specify which of the three JIT operating
@@ -989,34 +1022,34 @@ require it. Note also that if you request JIT compilation only for partial
matching (for example, jit=2) but do not set the partial modifier on a
subject line, that match will not use JIT code because none was compiled for
non-partial matching.
-
-
+
+
If JIT compilation is successful, the compiled JIT code will automatically be
used when an appropriate type of match is run, except when incompatible
run-time options are specified. For more details, see the
pcre2jit
documentation. See also the jitstack modifier below for a way of
setting the size of the JIT stack.
-
-
+
+
If the jitfast modifier is specified, matching is done using the JIT
"fast path" interface, pcre2_jit_match(), which skips some of the sanity
checks that are done by pcre2_match(), and of course does not work when
JIT is not supported. If jitfast is specified without jit, jit=7 is
assumed.
-
-
+
+
If the jitverify modifier is specified, information about the compiled
pattern shows whether JIT compilation was or was not successful. If
jitverify is specified without jit, jit=7 is assumed. If JIT
compilation is successful when jitverify is set, the text "(JIT)" is
added to the first output line after a match or non match when JIT-compiled
code was actually used in the match.
-
-
+
+
Setting a locale
-
-
+
+
The locale modifier must specify the name of a locale, for example:
/pattern/locale=fr_FR
@@ -1028,11 +1061,11 @@ are used when matching the following subject lines. The locale modifier
applies only to the pattern on which it appears, but can be given in a
#pattern command if a default is needed. Setting a locale and alternate
character tables are mutually exclusive.
-
-
+
+
Showing pattern memory
-
-
+
+
The memory modifier causes the size in bytes of the memory used to hold
the compiled pattern to be output. This does not include the size of the
pcre2_code block; it is just the actual compiled data. If the pattern is
@@ -1043,40 +1076,40 @@ also output. Here is an example:
Memory allocation (code space): 21
Memory allocation (JIT code): 1910
-
-
-
+
+
+
Limiting nested parentheses
-
-
+
+
The parens_nest_limit modifier sets a limit on the depth of nested
parentheses in a pattern. Breaching the limit causes a compilation error.
The default for the library is set when PCRE2 is built, but pcre2test
sets its own default of 220, which is required for running the standard test
suite.
-
-
+
+
Limiting the pattern length
-
-
+
+
The max_pattern_length modifier sets a limit, in code units, to the
length of pattern that pcre2_compile() will accept. Breaching the limit
causes a compilation error. The default is the largest number a PCRE2_SIZE
variable can hold (essentially unlimited).
-
-
+
+
Limiting the size of a compiled pattern
-
-
+
+
The max_pattern_compiled_length modifier sets a limit, in bytes, to the
amount of memory used by a compiled pattern. Breaching the limit causes a
compilation error. The default is the largest number a PCRE2_SIZE variable can
hold (essentially unlimited).
-
-
+
+
Using the POSIX wrapper API
-
-
+
+
The posix and posix_nosub modifiers cause pcre2test to call
PCRE2 via the POSIX wrapper API rather than its native API. When
posix_nosub is used, the POSIX option REG_NOSUB is passed to
@@ -1101,21 +1134,21 @@ is passed to regerror() in the event of a compilation error. For example:
This provides a means of testing the behaviour of regerror() when the
buffer is too small for the error message. If this modifier has not been set, a
large buffer is used.
-
-
+
+
The aftertext and allaftertext subject modifiers work as described
below. All other modifiers are either ignored, with a warning message, or cause
an error.
-
-
+
+
The pattern is passed to regcomp() as a zero-terminated string by
default, but if the use_length or hex modifiers are set, the
REG_PEND extension is used to pass it by length.
-
-
+
+
Testing the stack guard feature
-
-
+
+
The stackguard modifier is used to test the use of
pcre2_set_compile_recursion_guard(), a function that is provided to
enable stack availability to be checked during compilation (see the
@@ -1126,11 +1159,11 @@ callback from pcre2_compile() to a local function. The argument it
receives is the current nesting parenthesis depth; if this is greater than the
value given by the modifier, non-zero is returned, causing the compilation to
be aborted.
-
-
+
+
Using alternative character tables
-
-
+
+
The value specified for the tables modifier must be one of the digits 0,
1, 2, or 3. It causes a specific set of built-in character tables to be passed
to pcre2_compile(). This is used in the PCRE2 tests to check behaviour
@@ -1146,11 +1179,11 @@ In tables 2, some characters whose codes are greater than 128 are identified as
letters, digits, spaces, etc. Tables 3 can be used only after a
#loadtables command has loaded them from a binary file. Setting alternate
character tables and a locale are mutually exclusive.
-
-
+
+
Setting certain match controls
-
-
+
+
The following modifiers are really subject modifiers, and are described under
"Subject Modifiers" below. However, they may be included in a pattern's
modifier list, in which case they are applied to every subject line that is
@@ -1167,7 +1200,8 @@ process.
heapframes_size show match data heapframes size
jitstack=<n> set size of JIT stack
mark show mark values
- replace=<string> specify a replacement string
+ null_substitute_match_data substitute with NULL match data
+ replace=<str> specify a replacement string
startchar show starting character when relevant
substitute_callout use substitution callouts
substitute_case_callout use substitution case callouts
@@ -1183,20 +1217,20 @@ process.
These modifiers may not appear in a #pattern command. If you want them as
defaults, set them in a #subject command.
-
-
+
+
Specifying literal subject lines
-
-
+
+
If the subject_literal modifier is present on a pattern, all the subject
lines that it matches are taken as literal strings, with no interpretation of
backslashes. It is not possible to set subject modifiers on such lines, but any
that are set as defaults by a #subject command are recognized.
-
-
+
+
Saving a compiled pattern
-
-
+
+
When a pattern with the push modifier is successfully compiled, it is
pushed onto a stack of compiled patterns, and pcre2test expects the next
line to contain a new pattern (or a command) instead of a subject line. This
@@ -1213,11 +1247,11 @@ are ignored (for the stacked copy), with a warning message, except for
replace, which causes an error. Note that jitverify, which is
allowed, does not carry through to any subsequent matching that uses a stacked
pattern.
-
-
+
+
Testing foreign pattern conversion
-
-
+
+
The experimental foreign pattern conversion functions in PCRE2 can be tested by
setting the convert modifier. Its argument is a colon-separated list of
options, which set the equivalent option for the pcre2_pattern_convert()
@@ -1237,27 +1271,27 @@ result is reflected in the output and then passed to pcre2_compile(). The
normal utf and no_utf_check options, if set, cause the
PCRE2_CONVERT_UTF and PCRE2_CONVERT_NO_UTF_CHECK options to be passed to
pcre2_pattern_convert().
-
-
+
+
By default, the conversion function is allowed to allocate a buffer for its
output. However, if the convert_length modifier is set to a value greater
than zero, pcre2test passes a buffer of the given length. This makes it
possible to test the length check.
-
-
+
+
The convert_glob_escape and convert_glob_separator modifiers can be
used to specify the escape and separator characters for glob processing,
overriding the defaults, which are operating-system dependent.
-
-
SUBJECT MODIFIERS
-
+
+
+
The modifiers that can appear in subject lines and the #subject
command are of two types.
-
-
+
+
Setting match options
-
-
+
+
The following modifiers set options for pcre2_match() or
pcre2_dfa_match(). See
pcre2api
@@ -1280,15 +1314,15 @@ for a description of their effects.
The partial matching modifiers are provided with abbreviations because they
appear frequently in tests.
-
-
+
+
If the posix or posix_nosub modifier was present on the pattern,
causing the POSIX wrapper API to be used, the only option-setting modifiers
that have any effect are notbol, notempty, and noteol,
causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively, to be passed to
regexec(). The other modifiers are ignored, with a warning message.
-
-
+
+
There is one additional modifier that can be used with the POSIX wrapper. It is
ignored (with a warning) if used for non-POSIX matching.
@@ -1302,11 +1336,11 @@ the subject string. For more detail of REG_STARTEND, see the
documentation. If the subject string contains binary zeros (coded as escapes
such as \x{00} because pcre2test does not support actual binary zeros in
its input), you must use posix_startend to specify its length.
-
-
+
+
Setting match controls
-
-
+
+
The following modifiers affect the matching process or request additional
information. Some of them may also be specified on a pattern line (see above),
in which case they apply to every subject line that is matched against that
@@ -1342,11 +1376,12 @@ pattern, but can be overridden by modifiers on the subject.
null_context match with a NULL context
null_replacement substitute with NULL replacement
null_subject match with NULL subject
+ null_substitute_match_data substitute with NULL match data
offset=<n> set starting offset
offset_limit=<n> set offset limit
ovector=<n> set size of output vector
recursion_limit=<n> obsolete synonym for depth_limit
- replace=<string> specify a replacement string
+ replace=<str> specify a replacement string
startchar show startchar when relevant
startoffset=<n> same as offset=<n>
substitute_callout use substitution callouts
@@ -1358,6 +1393,7 @@ pattern, but can be overridden by modifiers on the subject.
substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY
substitute_skip=<n> skip substitution number n
substitute_stop=<n> skip substitution number n and greater
+ substitute_subject=<str> specify a different subject for substitution
substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY
zero_terminate pass the subject as zero-terminated
@@ -1366,11 +1402,11 @@ The effects of these modifiers are described in the following sections. When
matching via the POSIX wrapper API, the aftertext, allaftertext,
and ovector subject modifiers work as described below. All other
modifiers are either ignored, with a warning message, or cause an error.
-
-
+
+
Showing more text
-
-
+
+
The aftertext modifier requests that as well as outputting the part of
the subject string that matched the entire pattern, pcre2test should in
addition output the remainder of the subject string. This is useful for tests
@@ -1378,8 +1414,8 @@ where the subject contains multiple copies of the same substring. The
allaftertext modifier requests the same action for captured substrings as
well as the main matched substring. In each case the remainder is output on the
following line with a plus character following the capture number.
-
-
+
+
The allusedtext modifier requests that all the text that was consulted
during a successful pattern match by the interpreter should be shown, for both
full and partial matches. This feature is not supported for JIT matching, and
@@ -1402,8 +1438,8 @@ The first, complete match shows that the matched string is "abc", with the
preceding and following strings "pqr" and "xyz" having been consulted during
the match (when processing the assertions). The partial match can indicate only
the preceding string.
-
-
+
+
The startchar modifier requests that the starting character for the match
be indicated, if it is different to the start of the matched string. The only
time when this occurs is when \K has been processed as part of the match. In
@@ -1418,11 +1454,11 @@ under the earlier characters. For example:
Unlike allusedtext, the startchar modifier can be used with JIT.
However, these two modifiers are mutually exclusive.
-
-
+
+
Showing the value of all capture groups
-
-
+
+
The allcaptures modifier requests that the values of all potential
captured parentheses be output after a match. By default, only those up to the
highest one actually used in the match are output (corresponding to the return
@@ -1430,11 +1466,11 @@ code from pcre2_match()). Groups that did not take part in the match
are output as "<unset>". This modifier is not relevant for DFA matching (which
does no capturing) and does not apply when replace is specified; it is
ignored, with a warning message, if present.
-
-
+
+
Showing the entire ovector, for all outcomes
-
-
+
+
The allvector modifier requests that the entire ovector be shown,
whatever the outcome of the match. Compare allcaptures, which shows only
up to the maximum number of capture groups for the pattern, and then only for a
@@ -1447,11 +1483,11 @@ applies to all groups after the maximum capture group for the pattern. In other
cases it applies to the entire ovector. After a partial match, the first two
elements are the only ones that should be set. After a DFA match, the amount of
ovector that is used depends on the number of matches that were found.
-
-
+
+
Testing pattern callouts
-
-
+
+
A callout function is supplied when pcre2test calls the library matching
functions, unless callout_none is specified. Its behaviour can be
controlled by various modifiers listed above whose names begin with
@@ -1460,11 +1496,11 @@ controlled by various modifiers listed above whose names begin with
Testing callouts from pcre2_substitute() is described separately in
"Testing the substitution function"
below.
-
-
+
+
Finding all matches in a string
-
-
+
+
Searching for all possible matches within a subject can be requested by the
global or altglobal modifier. After finding a match, the matching
function is called again to search the remainder of the subject. The difference
@@ -1474,21 +1510,17 @@ to start searching at a new point within the entire string (which is what Perl
does), whereas the latter passes over a shortened subject. This makes a
difference to the matching process if the pattern begins with a lookbehind
assertion (including \b or \B).
-
-
+
+
If an empty string is matched, the next match is done with the
-PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search for
-another, non-empty, match at the same point in the subject. If this match
-fails, the start offset is advanced, and the normal match is retried. This
-imitates the way Perl handles such cases when using the /g modifier or
-the split() function. Normally, the start offset is advanced by one
-character, but if the newline convention recognizes CRLF as a newline, and the
-current character is CR followed by LF, an advance of two characters occurs.
-
-
+PCRE2_NOTEMPTY_ATSTART flag set, in order to search for another, non-empty,
+match at the same point in the subject. This imitates the way Perl handles such
+cases when using the /g modifier or the split() function.
+
+
Testing substring extraction functions
-
-
+
+
The copy and get modifiers can be used to test the
pcre2_substring_copy_xxx() and pcre2_substring_get_xxx() functions.
They can be given more than once, and each can specify a capture group name or
@@ -1499,43 +1531,43 @@ number, for example:
If the #subject command is used to set default copy and/or get lists,
these can be unset by specifying a negative number to cancel all numbered
groups and an empty name to cancel all named groups.
-
-
+
+
The getall modifier tests pcre2_substring_list_get(), which
extracts all captured substrings.
-
-
+
+
If the subject line is successfully matched, the substrings extracted by the
convenience functions are output with C, G, or L after the string number
instead of a colon. This is in addition to the normal full list. The string
length (that is, the return from the extraction function) is given in
parentheses after each substring, followed by the name when the extraction was
by name.
-
-
+
+
Testing the substitution function
-
-
+
+
If the replace modifier is set, the pcre2_substitute() function is
called instead of one of the matching functions (or after one call of
pcre2_match() in the case of PCRE2_SUBSTITUTE_MATCHED). Note that
replacement strings cannot contain commas, because a comma signifies the end of
a modifier. This is not thought to be an issue in a test program.
-
-
+
+
Specifying a completely empty replacement string disables this modifier.
However, it is possible to specify an empty replacement by providing a buffer
length, as described below, for an otherwise empty replacement.
-
-
+
+
Unlike subject strings, pcre2test does not process replacement strings
for escape sequences. In UTF mode, a replacement string is checked to see if it
is a valid UTF-8 string. If so, it is correctly converted to a UTF string of
the appropriate code unit width. If it is not a valid UTF-8 string, the
individual code units are copied directly. This provides a means of passing an
invalid UTF-8 string for testing purposes.
-
-
+
+
The following modifiers set options (in additional to the normal match options)
for pcre2_substitute():
@@ -1551,8 +1583,8 @@ for pcre2_substitute():
See the
pcre2api
documentation for details of these options.
-
-
+
+
After a successful substitution, the modified string is output, preceded by the
number of replacements. This may be zero if there were no matches. Here is a
simple example of a substitution test:
@@ -1574,7 +1606,7 @@ character. Here is an example that tests the edge case:
123abc123\=replace=[10]XYZ
1: 123XYZ123
123abc123\=replace=[9]XYZ
- Failed: error -47: no more memory
+ Failed: error -48: no more memory
The default action of pcre2_substitute() is to return
PCRE2_ERROR_NOMEMORY when the output buffer is too small. However, if the
@@ -1587,16 +1619,23 @@ for the trailing zero) as part of the error message. For example:
/abc/substitute_overflow_length
123abc123\=replace=[9]XYZ
- Failed: error -47: no more memory: 10 code units are needed
+ Failed: error -48: no more memory: 10 code units are needed
A replacement string is ignored with POSIX and DFA matching. Specifying partial
matching provokes an error return ("bad option value") from
pcre2_substitute().
-
-
+
+
+The substitute_subject modifier may be used to test the use of the PCRE2
+API, in which a client calls pcre2_match() followed by pcre2_substitute()
+with PCRE2_SUBSTITUTE_MATCHED, but the client performs an unexpected and
+unsupported modification of the subject buffer in-place, in between the match
+and substitution.
+
+
Testing substitute callouts
-
-
+
+
If the substitute_callout modifier is set, a substitution callout
function is set up. The null_context modifier must not be set, because
the address of the callout function is passed in a match context. When the
@@ -1614,8 +1653,8 @@ parenthesized number is the number of pairs that are set in the ovector (that
is, one more than the number of capturing groups that were set). Then are
listed the offsets of the old substring, its contents, and the same for the
replacement.
-
-
+
+
By default, the substitution callout function returns zero, which accepts the
replacement and causes matching to continue if /g was used. Two further
modifiers can be used to test other return values. If substitute_skip is
@@ -1635,26 +1674,26 @@ either of them are set, substitute_callout is assumed. For example:
If both are set for the same number, stop takes precedence. Only a single skip
or stop is supported, which is sufficient for testing that the feature works.
-
-
+
+
Testing substitute case callouts
-
-
+
+
If the substitute_case_callout modifier is set, a substitution
case callout function is set up. The callout function is called for each
substituted chunk which is to be case-transformed.
-
-
+
+
The callout function passed is a fixed function with implementation for certain
behaviours: inputs which shrink when case-transformed; inputs which grow; inputs
with distinct upper/lower/titlecase forms. The characters which are not
special-cased for testing purposes are left unmodified, as if they are caseless
characters.
-
-
+
+
Setting the JIT stack size
-
-
+
+
The jitstack modifier provides a way of setting the maximum stack size
that is used by the just-in-time optimization code. It is ignored if JIT
optimization is not being used. The value is a number of kibibytes (units of
@@ -1662,19 +1701,19 @@ optimization is not being used. The value is a number of kibibytes (units of
that is larger than the default is necessary only for very complicated
patterns. If jitstack is set non-zero on a subject line it overrides any
value that was set on the pattern.
-
-
+
+
Setting heap, match, and depth limits
-
-
+
+
The heap_limit, match_limit, and depth_limit modifiers set
the appropriate limits in the match context. These values are ignored when the
find_limits or find_limits_noheap modifier is specified.
-
-
+
+
Finding minimum limits
-
-
+
+
If the find_limits modifier is present on a subject line, pcre2test
calls the relevant matching function several times, setting different values in
the match context via pcre2_set_heap_limit(),
@@ -1685,22 +1724,22 @@ modifier, find_limits_noheap, omits the heap limit. This is used in the
standard tests, because the minimum heap limit varies between systems. If JIT
is being used, only the match limit is relevant, and the other two are
automatically omitted.
-
-
+
+
When using this modifier, the pattern should not contain any limit settings
such as (*LIMIT_MATCH=...) within it. If such a setting is present and is
lower than the minimum matching value, the minimum value cannot be found
because pcre2_set_match_limit() etc. are only able to reduce the value of
an in-pattern limit; they cannot increase it.
-
-
+
+
For non-DFA matching, the minimum depth_limit number is a measure of how
much nested backtracking happens (that is, how deeply the pattern's tree is
searched). In the case of DFA matching, depth_limit controls the depth of
recursive calls of the internal function that is used for handling pattern
recursion, lookaround assertions, and atomic groups.
-
-
+
+
For non-DFA matching, the match_limit number is a measure of the amount
of backtracking that takes place, and learning the minimum value can be
instructive. For most simple matches, the number is quite small, but for
@@ -1709,25 +1748,25 @@ very quickly with increasing length of subject string. In the case of DFA
matching, match_limit controls the total number of calls, both recursive
and non-recursive, to the internal matching function, thus controlling the
overall amount of computing resource that is used.
-
-
+
+
For both kinds of matching, the heap_limit number, which is in kibibytes
(units of 1024 bytes), limits the amount of heap memory used for matching.
-
-
+
+
Showing MARK names
-
-
+
+
The mark modifier causes the names from backtracking control verbs that
are returned from calls to pcre2_match() to be displayed. If a mark is
returned for a match, non-match, or partial match, pcre2test shows it.
For a match, it is on a line by itself, tagged with "MK:". Otherwise, it
is added to the non-match message.
-
-
+
+
Showing memory usage
-
-
+
+
The memory modifier causes pcre2test to log the sizes of all heap
memory allocation and freeing calls that occur during a call to
pcre2_match() or pcre2_dfa_match(). In the latter case, heap memory
@@ -1736,52 +1775,52 @@ allocation on the stack, so in many cases there will be no output. No heap
memory is allocated during matching with JIT. For this modifier to work, the
null_context modifier must not be set on both the pattern and the
subject, though it can be set on one or the other.
-
-
+
+
Showing the heap frame overall vector size
-
-
+
+
The heapframes_size modifier is relevant for matches using
pcre2_match() without JIT. After a match has run (whether successful or
not) the size, in bytes, of the allocated heap frames vector that is left
attached to the match data block is shown. If the matching action involved
several calls to pcre2_match() (for example, global matching or for
timing) only the final value is shown.
-
-
+
+
This modifier is ignored, with a warning, for POSIX or DFA matching. JIT
matching does not use the heap frames vector, so the size is always zero,
unless there was a previous non-JIT match. Note that specifing a size of zero
for the output vector (see below) causes pcre2test to free its match data
block (and associated heap frames vector) and allocate a new one.
-
-
+
+
Setting a starting offset
-
-
+
+
The offset modifier sets an offset in the subject string at which
matching starts. Its value is a number of code units, not characters.
-
-
+
+
Setting an offset limit
-
-
+
+
The offset_limit modifier sets a limit for unanchored matches. If a match
cannot be found starting at or before this offset in the subject, a "no match"
return is given. The data value is a number of code units, not characters. When
this modifier is used, the use_offset_limit modifier must have been set
for the pattern; if not, an error is generated.
-
-
+
+
Setting the size of the output vector
-
-
+
+
The ovector modifier applies only to the subject line in which it
appears, though of course it can also be used to set a default in a
#subject command. It specifies the number of pairs of offsets that are
available for storing matching information. The default is 15.
-
-
+
+
A value of zero is useful when testing the POSIX API because it causes
regexec() to be called with a NULL capture vector. When not testing the
POSIX API, a value of zero is used to cause
@@ -1789,25 +1828,25 @@ POSIX API, a value of zero is used to cause
new match block of exactly the right size for the pattern. (It is not possible
to create a match block with a zero-length ovector; there is always at least
one pair of offsets.) The old match data block is freed.
-
-
+
+
Passing the subject as zero-terminated
-
-
+
+
By default, the subject string is passed to a native API matching function with
its correct length. In order to test the facility for passing a zero-terminated
string, the zero_terminate modifier is provided. It causes the length to
be passed as PCRE2_ZERO_TERMINATED. When matching via the POSIX interface,
this modifier is ignored, with a warning.
-
-
+
+
When testing pcre2_substitute(), this modifier also has the effect of
passing the replacement string as zero-terminated.
-
-
+
+
Passing a NULL context, subject, or replacement
-
-
+
+
Normally, pcre2test passes a context block to pcre2_match(),
pcre2_dfa_match(), pcre2_jit_match() or pcre2_substitute().
If the null_context modifier is set, however, NULL is passed. This is for
@@ -1815,14 +1854,14 @@ testing that the matching and substitution functions behave correctly in this
case (they use default values). This modifier cannot be used with the
find_limits, find_limits_noheap, or substitute_callout
modifiers.
-
-
+
+
Similarly, for testing purposes, if the null_subject or
null_replacement modifier is set, the subject or replacement string
pointers are passed as NULL, respectively, to the relevant functions.
-
-
THE ALTERNATIVE MATCHING FUNCTION
-
+
+
+
By default, pcre2test uses the standard PCRE2 matching function,
pcre2_match() to match each subject line. PCRE2 also supports an
alternative matching function, pcre2_dfa_match(), which operates in a
@@ -1830,19 +1869,19 @@ different way, and has some restrictions. The differences between the two
functions are described in the
pcre2matching
documentation.
-
-
+
+
If the dfa modifier is set, the alternative matching function is used.
This function finds all possible matches at a given point in the subject. If,
however, the dfa_shortest modifier is set, processing stops after the
first match is found. This is always the shortest possible match.
-
-
DEFAULT OUTPUT FROM pcre2test
-
+
+
+
This section describes the output when the normal matching function,
pcre2_match(), is being used.
-
-
+
+
When a match succeeds, pcre2test outputs the list of captured substrings,
starting with number 0 for the string that matched the whole pattern.
Otherwise, it outputs "No match" when the return is PCRE2_ERROR_NOMATCH, or
@@ -1851,8 +1890,8 @@ return is PCRE2_ERROR_PARTIAL. (Note that this is the
entire substring that was inspected during the partial match; it may include
characters before the actual match start if a lookbehind assertion, \K, \b,
or \B was involved.)
-
-
+
+
For any other return, pcre2test outputs the PCRE2 negative error number
and a short descriptive phrase. If the error is a failed UTF string check, the
code unit offset of the start of the failing character is also output. Here is
@@ -1913,16 +1952,16 @@ modifier is past the end of the subject string):
re> /xyz/
data> xyz\=offset=4
Error -24 (bad offset value)
-
-
-
+
+
+
Note that whereas patterns can be continued over several lines (a plain ">"
prompt is used for continuations), subject lines may not. However newlines can
be included in a subject by means of the \n escape (or \r, \r\n, etc.,
depending on the newline sequence setting).
-
-
OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION
-
+
+
+
When the alternative matching function, pcre2_dfa_match(), is used, the
output consists of a list of all the matches that start at the first point in
the subject where there is at least one match. For example:
@@ -1940,8 +1979,8 @@ partially matching substring. Note that this is the entire substring that was
inspected during the partial match; it may include characters before the actual
match start if a lookbehind assertion, \b, or \B was involved. (\K is not
supported for DFA matching.)
-
-
+
+
If global matching is requested, the search for further matches resumes
at the end of the longest match. For example:
@@ -1956,9 +1995,9 @@ at the end of the longest match. For example:
The alternative matching function does not support substring capture, so the
modifiers that are concerned with captured substrings are not relevant.
-
-
RESTARTING AFTER A PARTIAL MATCH
-
+
+
+
When the alternative matching function has given the PCRE2_ERROR_PARTIAL
return, indicating that the subject partially matched the pattern, you can
restart the match with additional subject data by means of the
@@ -1973,19 +2012,19 @@ restart the match with additional subject data by means of the
For further information about partial matching, see the
pcre2partial
documentation.
-
-
CALLOUTS
-
+
+
+
If the pattern contains any callout requests, pcre2test's callout
function is called during matching unless callout_none is specified. This
works with both matching functions, and with JIT, though there are some
differences in behaviour. The output for callouts with numerical arguments and
those with string arguments is slightly different.
-
-
+
+
Callouts with numerical arguments
-
-
+
+
By default, the callout function displays the callout number, the start and
current positions in the subject text at the callout time, and the next pattern
item to be tested. For example:
@@ -1999,8 +2038,8 @@ the seventh character, and when the next pattern item was \d. Just
one circumflex is output if the start and current positions are the same, or if
the current position precedes the start position, which can happen if the
callout is in a lookbehind assertion.
-
-
+
+
Callouts numbered 255 are assumed to be automatic callouts, inserted as a
result of the auto_callout pattern modifier. In this case, instead of
showing the callout number, the offset in the pattern, preceded by a plus, is
@@ -2032,11 +2071,11 @@ a change of latest mark is passed to the callout function. For example:
The mark changes between matching "a" and "b", but stays the same for the rest
of the match, so nothing more is output. If, as a result of backtracking, the
mark reverts to being unset, the text "<unset>" is output.
-
-
+
+
Callouts with string arguments
-
-
+
+
The output for a callout with a string argument is similar, except that instead
of outputting a callout number before the position indicators, the callout
string and its offset in the pattern string are output before the reflection of
@@ -2053,27 +2092,27 @@ example:
^ ^ e
0: abcdef
-
-
-
+
+
+
Callout modifiers
-
-
+
+
The callout function in pcre2test returns zero (carry on matching) by
default, but you can use a callout_fail modifier in a subject line to
change this and other parameters of the callout (see below).
-
-
+
+
If the callout_capture modifier is set, the current captured groups are
output when a callout occurs. This is useful only for non-DFA matching, as
pcre2_dfa_match() does not support capturing, so no captures are ever
shown.
-
-
+
+
The normal callout output, showing the callout number or pattern offset (as
described above) is suppressed if the callout_no_where modifier is set.
-
-
+
+
When using the interpretive matching function pcre2_match() without JIT,
setting the callout_extra modifier causes additional output from
pcre2test's callout function to be generated. For the first callout in a
@@ -2123,15 +2162,15 @@ an immediate "no match", without any callouts, because the starting
optimization fails to find "b" in the subject, which it knows must be present
for any match. If no_auto_possess is not used, the "a+" item is turned
into "a++", which reduces the number of backtracks.
-
-
+
+
The callout_extra modifier has no effect if used with the DFA matching
function, or with JIT.
-
-
+
+
Return values from callouts
-
-
+
+
The default return from the callout function is zero, which allows matching to
continue. The callout_fail modifier can be given one or two numbers. If
there is only one number, 1 is returned instead of 0 (causing matching to
@@ -2142,35 +2181,35 @@ PCRE2_ERROR_CALLOUT is returned, causing the entire matching process to be
aborted. If both these modifiers are set for the same callout number,
callout_error takes precedence. Note that callouts with string arguments
are always given the number zero.
-
-
+
+
The callout_data modifier can be given an unsigned or a negative number.
This is set as the "user data" that is passed to the matching function, and
passed back when the callout function is invoked. Any value other than zero is
used as a return from pcre2test's callout function.
-
-
+
+
Inserting callouts can be helpful when using pcre2test to check
complicated regular expressions. For further information about callouts, see
the
pcre2callout
documentation.
-
-
NON-PRINTING CHARACTERS
-
+
+
+
When pcre2test is outputting text in the compiled version of a pattern,
bytes other than 32-126 are always treated as non-printing characters and are
therefore shown as hex escapes.
-
-
+
+
When pcre2test is outputting text that is a matched part of a subject
string, it behaves in the same way, unless a different locale has been set for
the pattern (using the locale modifier). In this case, the
isprint() function is used to distinguish printing and non-printing
characters.
-
-
SAVING AND RESTORING COMPILED PATTERNS
-
+
+
+
It is possible to save compiled patterns on disc or elsewhere, and reload them
later, subject to a number of restrictions. JIT data cannot be saved. The host
on which the patterns are reloaded must be running the same version of PCRE2,
@@ -2180,20 +2219,20 @@ serialized, that is, converted to a stream of bytes. A single byte stream may
contain any number of compiled patterns, but they must all use the same
character tables. A single copy of the tables is included in the byte stream
(its size is 1088 bytes).
-
-
+
+
The functions whose names begin with pcre2_serialize_ are used
for serializing and de-serializing. They are described in the
pcre2serialize
documentation. In this section we describe the features of pcre2test that
can be used to test these functions.
-
-
+
+
Note that "serialization" in PCRE2 does not convert compiled patterns to an
abstract format like Java or .NET. It just makes a reloadable byte code stream.
Hence the restrictions on reloading mentioned above.
-
-
+
+
In pcre2test, when a pattern with push modifier is successfully
compiled, it is pushed onto a stack of compiled patterns, and pcre2test
expects the next line to contain a new pattern (or command) instead of a
@@ -2204,8 +2243,8 @@ be compiled and retained. These modifiers are incompatible with posix,
and control modifiers that act at match time are ignored (with a message) for
the stacked patterns. The jitverify modifier applies only at compile
time.
-
-
+
+
The command
#save <filename>
@@ -2241,30 +2280,30 @@ reloads two patterns.
If jitverify is used with #pop, it does not automatically imply
jit, which is different behaviour from when it is used on a pattern.
-
-
+
+
The #popcopy command is analogous to the pushcopy modifier in that it
makes current a copy of the topmost stack pattern, leaving the original still
on the stack.
-
-
SEE ALSO
-
+
+
+
pcre2(3), pcre2api(3), pcre2callout(3),
pcre2jit, pcre2matching(3), pcre2partial(d),
pcre2pattern(3), pcre2serialize(3).
-
-
AUTHOR
-
+
+
+
Philip Hazel
Retired from University Computing Service
Cambridge, England.
-
-
REVISION
-
-Last updated: 26 December 2024
+
+
+
+Last updated: 12 October 2025
Copyright © 1997-2024 University of Cambridge.
diff --git a/doc/html/pcre2unicode.html b/doc/html/pcre2unicode.html
index 5b42532..8c278ae 100644
--- a/doc/html/pcre2unicode.html
+++ b/doc/html/pcre2unicode.html
@@ -12,18 +12,18 @@ This page is part of the PCRE2 HTML documentation. It was generated
automatically from the original man page. If there is any nonsense in it,
please consult the man page, in case the conversion went wrong.
-
+
UNICODE AND UTF SUPPORT
-
-
+
+
PCRE2 is normally built with Unicode support, though if you do not need it, you
can build it without, in which case the library will be smaller. With Unicode
support, PCRE2 has knowledge of Unicode character properties and can process
strings of text in UTF-8, UTF-16, and UTF-32 format (depending on the code unit
width), but this is not the default. Unless specifically requested, PCRE2
treats each code unit in a string as one character.
-
-
+
+
There are two ways of telling PCRE2 to switch to UTF mode, where characters may
consist of more than one code unit and the range of values is constrained. The
program can call
@@ -32,22 +32,22 @@ with the PCRE2_UTF option, or the pattern may start with the sequence (*UTF).
However, the latter facility can be locked out by the PCRE2_NEVER_UTF option.
That is, the programmer can prevent the supplier of the pattern from switching
to UTF mode.
-
-
+
+
Note that the PCRE2_MATCH_INVALID_UTF option (see
below)
forces PCRE2_UTF to be set.
-
-
+
+
In UTF mode, both the pattern and any subject strings that are matched against
it are treated as UTF strings instead of strings of individual one-code-unit
characters. There are also some other changes to the way characters are
handled, as documented below.
-
-
+
+
UNICODE PROPERTY SUPPORT
-
-
+
+
When PCRE2 is built with Unicode support, the escape sequences \p{..},
\P{..}, and \X can be used. This is not dependent on the PCRE2_UTF setting.
The Unicode properties that can be tested are a subset of those that Perl
@@ -55,8 +55,8 @@ supports. Currently they are limited to the general category properties such as
Lu for an upper case letter or Nd for a decimal number, the derived properties
Any and Lc (synonym L&), the Unicode script names such as Arabic or Han,
Bidi_Class, Bidi_Control, and a few binary properties.
-
-
+
+
The full lists are given in the
pcre2pattern
and
@@ -65,34 +65,34 @@ documentation. In general, only the short names for properties are supported.
For example, \p{L} matches a letter. Its longer synonym, \p{Letter}, is not
supported. Furthermore, in Perl, many properties may optionally be prefixed by
"Is", for compatibility with Perl 5.6. PCRE2 does not support this.
-
-
+
+
WIDE CHARACTERS AND UTF MODES
-
-
+
+
Code points less than 256 can be specified in patterns by either braced or
unbraced hexadecimal escape sequences (for example, \x{b3} or \xb3). Larger
values have to use braced sequences. Unbraced octal code points up to \777 are
also recognized; larger ones can be coded using \o{...}.
-
-
+
+
The escape sequence \N{U+<hex digits>} is recognized as another way of
specifying a Unicode character by code point in a UTF mode. It is not allowed
in non-UTF mode.
-
-
+
+
In UTF mode, repeat quantifiers apply to complete UTF characters, not to
individual code units.
-
-
+
+
In UTF mode, the dot metacharacter matches one UTF character instead of a
single code unit.
-
-
+
+
In UTF mode, capture group names are not restricted to ASCII, and may contain
any Unicode letters and decimal digits, as well as underscore.
-
-
+
+
The escape sequence \C can be used to match a single code unit in UTF mode,
but its use can lead to some strange effects because it breaks up multi-unit
characters (see the description of \C in the
@@ -100,8 +100,8 @@ characters (see the description of \C in the
documentation). For this reason, there is a build-time option that disables
support for \C completely. There is also a less draconian compile-time option
for locking out the use of \C when a pattern is compiled.
-
-
+
+
The use of \C is not supported by the alternative matching function
pcre2_dfa_match() when in UTF-8 or UTF-16 mode, that is, when a character
may consist of more than one code unit. The use of \C in these modes provokes
@@ -109,8 +109,8 @@ a match-time error. Also, the JIT optimization does not support \C in these
modes. If JIT optimization is requested for a UTF-8 or UTF-16 pattern that
contains \C, it will not succeed, and so when pcre2_match() is called,
the matching will be carried out by the interpretive function.
-
-
+
+
The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly test
characters of any code value, but, by default, the characters that PCRE2
recognizes as digits, spaces, or word characters remain the same set as in
@@ -127,21 +127,21 @@ that suppress this for individual escapes. For details see the section on
in the
pcre2pattern
documentation.
-
-
+
+
Like the escapes, characters that match the POSIX named character classes are
all low-valued characters unless the PCRE2_UCP option is set, but there is an
option to override this.
-
-
+
+
In contrast to the character escapes and character classes, the special
horizontal and vertical white space escapes (\h, \H, \v, and \V) do match
all the appropriate Unicode characters, whether or not PCRE2_UCP is set.
-
-
+
+
UNICODE CASE-EQUIVALENCE
-
-
+
+
If either PCRE2_UTF or PCRE2_UCP is set, upper/lower case processing makes use
of Unicode properties except for characters whose code points are less than 128
and that have at most two case-equivalent values. For these, a direct table
@@ -149,8 +149,8 @@ lookup is used for speed. A few Unicode characters such as Greek sigma have
more than two code points that are case-equivalent, and these are treated
specially. Setting PCRE2_UCP without PCRE2_UTF allows Unicode-style case
processing for non-UTF character encodings such as UCS-2.
-
-
+
+
There are two ASCII characters (S and K) that, in addition to their ASCII lower
case equivalents, have a non-ASCII one as well (long S and Kelvin sign).
Recognition of these non-ASCII characters as case-equivalent to their ASCII
@@ -166,9 +166,9 @@ ASCII or non-ASCII; there can be no mixing.
U+212A (Kelvin sign) only case-equivalent to itself
's' = 'S'
U+017F (long S) only case-equivalent to itself
-
-
-
+
+
+
One language family, Turkish and Azeri, has its own case-insensitivity rules,
which can be selected by setting PCRE2_EXTRA_TURKISH_CASING. This alters the
behaviour of the 'i', 'I', U+0130 (capital I with dot above), and U+0131
@@ -181,50 +181,50 @@ behaviour of the 'i', 'I', U+0130 (capital I with dot above), and U+0131
With PCRE2_EXTRA_TURKISH_CASING:
'i' = U+0130 (capital I with dot above)
U+0131 (small dotless i) = 'I'
-
-
-
+
+
+
It is not allowed to specify both PCRE2_EXTRA_CASELESS_RESTRICT and
PCRE2_EXTRA_TURKISH_CASING together.
-
-
+
+
From release 10.45 the Unicode letter properties Lu (upper case), Ll (lower
case), and Lt (title case) are all treated as Lc (cased letter) when caseless
matching is set by the PCRE2_CASELESS option or (?i) within the pattern.
-
-
+
+
SCRIPT RUNS
-
-
+
+
The pattern constructs (*script_run:...) and (*atomic_script_run:...), with
synonyms (*sr:...) and (*asr:...), verify that the string matched within the
parentheses is a script run. In concept, a script run is a sequence of
characters that are all from the same Unicode script. However, because some
scripts are commonly used together, and because some diacritical and other
marks are used with multiple scripts, it is not that simple.
-
-
+
+
Every Unicode character has a Script property, mostly with a value
corresponding to the name of a script, such as Latin, Greek, or Cyrillic. There
are also three special values:
-
-
+
+
"Unknown" is used for code points that have not been assigned, and also for the
surrogate code points. In the PCRE2 32-bit library, characters whose code
points are greater than the Unicode maximum (U+10FFFF), which are accessible
only in non-UTF mode, are assigned the Unknown script.
-
-
+
+
"Common" is used for characters that are used with many scripts. These include
punctuation, emoji, mathematical, musical, and currency symbols, and the ASCII
digits 0 to 9.
-
-
+
+
"Inherited" is used for characters such as diacritical marks that modify a
previous character. These are considered to take on the script of the character
that they modify.
-
-
+
+
Some Inherited characters are used with many scripts, but many of them are only
normally used with a small number of scripts. For example, U+102E0 (Coptic
Epact thousands mark) is used only with Arabic and Coptic. In order to make it
@@ -234,37 +234,37 @@ characters, the list contains just one script, the same one as the Script
property. However, for characters such as U+102E0 more than one Script is
listed. There are also some Common characters that have a single, non-Common
script in their Script Extension list.
-
-
+
+
The next section describes the basic rules for deciding whether a given string
of characters is a script run. Note, however, that there are some special cases
involving the Chinese Han script, and an additional constraint for decimal
digits. These are covered in subsequent sections.
-
-
+
+
Basic script run rules
-
-
+
+
A string that is less than two characters long is a script run. This is the
only case in which an Unknown character can be part of a script run. Longer
strings are checked using only the Script Extensions property, not the basic
Script property.
-
-
+
+
If a character's Script Extension property is the single value "Inherited", it
is always accepted as part of a script run. This is also true for the property
"Common", subject to the checking of decimal digits described below. All the
remaining characters in a script run must have at least one script in common in
their Script Extension lists. In set-theoretic terminology, the intersection of
all the sets of scripts must not be empty.
-
-
+
+
A simple example is an Internet name such as "google.com". The letters are all
in the Latin script, and the dot is Common, so this string is a script run.
However, the Cyrillic letter "o" looks exactly the same as the Latin "o"; a
string that looks the same, but with Cyrillic "o"s is not a script run.
-
-
+
+
More interesting examples involve characters with more than one script in their
Script Extension. Consider the following characters:
@@ -275,11 +275,11 @@ The first has the Script Extension list Arabic, Hanifi Rohingya, Syriac, and
Thaana; the second has just Arabic and Hanifi Rohingya. Both of them could
appear in script runs of either Arabic or Hanifi Rohingya. The first could also
appear in Syriac or Thaana script runs, but the second could not.
-
-
+
+
The Chinese Han script
-
-
+
+
The Chinese Han script is commonly used in conjunction with other scripts for
writing certain languages. Japanese uses the Hiragana and Katakana scripts
together with Han; Korean uses Hangul and Han; Taiwanese Mandarin uses Bopomofo
@@ -290,57 +290,57 @@ Han, or a mixture of Bopomofo and Han, but not, for example, a mixture of
Hangul and Bopomofo and Han. PCRE2 (like Perl) follows Unicode's Technical
Standard 39 ("Unicode Security Mechanisms", http://unicode.org/reports/tr39/)
in allowing such mixtures.
-
-
+
+
Decimal digits
-
-
+
+
Unicode contains many sets of 10 decimal digits in different scripts, and some
scripts (including the Common script) contain more than one set. Some of these
decimal digits them are visually indistinguishable from the common ASCII
digits. In addition to the script checking described above, if a script run
contains any decimal digits, they must all come from the same set of 10
adjacent characters.
-
-
+
+
VALIDITY OF UTF STRINGS
-
-
+
+
When the PCRE2_UTF option is set, the strings passed as patterns and subjects
are (by default) checked for validity on entry to the relevant functions. If an
invalid UTF string is passed, a negative error code is returned. The code unit
offset to the offending character can be extracted from the match data block by
calling pcre2_get_startchar(), which is used for this purpose after a UTF
error.
-
-
+
+
In some situations, you may already know that your strings are valid, and
therefore want to skip these checks in order to improve performance, for
example in the case of a long subject string that is being scanned repeatedly.
If you set the PCRE2_NO_UTF_CHECK option at compile time or at match time,
PCRE2 assumes that the pattern or subject it is given (respectively) contains
only valid UTF code unit sequences.
-
-
+
+
If you pass an invalid UTF string when PCRE2_NO_UTF_CHECK is set, the result
is undefined and your program may crash or loop indefinitely or give incorrect
results. There is, however, one mode of matching that can handle invalid UTF
subject strings. This is enabled by passing PCRE2_MATCH_INVALID_UTF to
pcre2_compile() and is discussed below in the next section. The rest of
this section covers the case when PCRE2_MATCH_INVALID_UTF is not set.
-
-
+
+
Passing PCRE2_NO_UTF_CHECK to pcre2_compile() just disables the UTF check
for the pattern; it does not also apply to subject strings. If you want to
disable the check for a subject string you must pass this same option to
pcre2_match() or pcre2_dfa_match().
-
-
+
+
UTF-16 and UTF-32 strings can indicate their endianness by special code knows
as a byte-order mark (BOM). The PCRE2 functions do not handle this, expecting
strings to be in host byte order.
-
-
+
+
Unless PCRE2_NO_UTF_CHECK is set, a UTF string is checked before any other
processing takes place. In the case of pcre2_match() and
pcre2_dfa_match() calls with a non-zero starting offset, the check is
@@ -352,33 +352,33 @@ Otherwise, it starts at the length of the longest lookbehind before the
starting offset, or at the start of the subject if there are not that many
characters before the starting offset. Note that the sequences \b and \B are
one-character lookbehinds.
-
-
+
+
In addition to checking the format of the string, there is a check to ensure
that all code points lie in the range U+0 to U+10FFFF, excluding the surrogate
area. The so-called "non-character" code points are not excluded because
Unicode corrigendum #9 makes it clear that they should not be.
-
-
+
+
Characters in the "Surrogate Area" of Unicode are reserved for use by UTF-16,
where they are used in pairs to encode code points with values greater than
0xFFFF. The code points that are encoded by UTF-16 pairs are available
independently in the UTF-8 and UTF-32 encodings. (In other words, the whole
surrogate thing is a fudge for UTF-16 which unfortunately messes up UTF-8 and
UTF-32.)
-
-
+
+
Setting PCRE2_NO_UTF_CHECK at compile time does not disable the error that is
given if an escape sequence for an invalid Unicode code point is encountered in
the pattern. If you want to allow escape sequences such as \x{d800} (a
surrogate code point) you can set the PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES extra
option. However, this is possible only in UTF-8 and UTF-32 modes, because these
values are not representable in UTF-16.
-
-
+
+
Errors in UTF-8 strings
-
-
+
+
The following negative error codes are given for invalid UTF-8 strings:
PCRE2_ERROR_UTF8_ERR1
@@ -442,111 +442,111 @@ character.
The first byte of a character has the value 0xfe or 0xff. These values can
never occur in a valid UTF-8 string.
-
-
+
+
Errors in UTF-16 strings
-
-
+
+
The following negative error codes are given for invalid UTF-16 strings:
PCRE2_ERROR_UTF16_ERR1 Missing low surrogate at end of string
PCRE2_ERROR_UTF16_ERR2 Invalid low surrogate follows high surrogate
PCRE2_ERROR_UTF16_ERR3 Isolated low surrogate
-
-
-
+
+
+
Errors in UTF-32 strings
-
-
+
+
The following negative error codes are given for invalid UTF-32 strings:
PCRE2_ERROR_UTF32_ERR1 Surrogate character (0xd800 to 0xdfff)
PCRE2_ERROR_UTF32_ERR2 Code point is greater than 0x10ffff
-
-
-
+
+
+
MATCHING IN INVALID UTF STRINGS
-
-
+
+
You can run pattern matches on subject strings that may contain invalid UTF
sequences if you call pcre2_compile() with the PCRE2_MATCH_INVALID_UTF
option. This is supported by pcre2_match(), including JIT matching, but
not by pcre2_dfa_match(). When PCRE2_MATCH_INVALID_UTF is set, it forces
PCRE2_UTF to be set as well. Note, however, that the pattern itself must be a
valid UTF string.
-
-
+
+
If you do not set PCRE2_MATCH_INVALID_UTF when calling pcre2_compile, and
you are not certain that your subject strings are valid UTF sequences, you
should not make use of the JIT "fast path" function pcre2_jit_match()
because it bypasses sanity checks, including the one for UTF validity. An
invalid string may cause undefined behaviour, including looping, crashing, or
giving the wrong answer.
-
-
+
+
Setting PCRE2_MATCH_INVALID_UTF does not affect what pcre2_compile()
generates, but if pcre2_jit_compile() is subsequently called, it does
generate different code. If JIT is not used, the option affects the behaviour
of the interpretive code in pcre2_match(). When PCRE2_MATCH_INVALID_UTF
is set at compile time, PCRE2_NO_UTF_CHECK is ignored at match time.
-
-
+
+
In this mode, an invalid code unit sequence in the subject never matches any
pattern item. It does not match dot, it does not match \p{Any}, it does not
even match negative items such as [^X]. A lookbehind assertion fails if it
encounters an invalid sequence while moving the current point backwards. In
other words, an invalid UTF code unit sequence acts as a barrier which no match
can cross.
-
-
+
+
You can also think of this as the subject being split up into fragments of
valid UTF, delimited internally by invalid code unit sequences. The pattern is
matched fragment by fragment. The result of a successful match, however, is
given as code unit offsets in the entire subject string in the usual way. There
are a few points to consider:
-
-
+
+
The internal boundaries are not interpreted as the beginnings or ends of lines
and so do not match circumflex or dollar characters in the pattern.
-
-
+
+
If pcre2_match() is called with an offset that points to an invalid
UTF-sequence, that sequence is skipped, and the match starts at the next valid
UTF character, or the end of the subject.
-
-
+
+
At internal fragment boundaries, \b and \B behave in the same way as at the
beginning and end of the subject. For example, a sequence such as \bWORD\b
would match an instance of WORD that is surrounded by invalid UTF code units.
-
-
+
+
Using PCRE2_MATCH_INVALID_UTF, an application can run matches on arbitrary
data, knowing that any matched strings that are returned are valid UTF. This
can be useful when searching for UTF text in executable or other binary files.
-
-
+
+
Note, however, that the 16-bit and 32-bit PCRE2 libraries process strings as
sequences of uint16_t or uint32_t code points. They cannot find valid UTF
sequences within an arbitrary string of bytes unless such sequences are
suitably aligned.
-
-
+
+
AUTHOR
-
-
+
+
Philip Hazel
Retired from University Computing Service
Cambridge, England.
-
-
+
+
REVISION
-
-
+
+
Last updated: 27 November 2024
Copyright © 1997-2024 University of Cambridge.
diff --git a/doc/pcre2-config.1 b/doc/pcre2-config.1
index 3b7a8c2..c119a11 100644
--- a/doc/pcre2-config.1
+++ b/doc/pcre2-config.1
@@ -1,4 +1,4 @@
-.TH PCRE2-CONFIG 1 "28 September 2014" "PCRE2 10.46"
+.TH PCRE2-CONFIG 1 "22 February 2025" "PCRE2 10.47"
.SH NAME
pcre2-config - program to return PCRE2 configuration
.SH SYNOPSIS
@@ -16,22 +16,23 @@ pcre2-config - program to return PCRE2 configuration
.sp
\fBpcre2-config\fP returns the configuration of the installed PCRE2 libraries
and the options required to compile a program to use them. Some of the options
-apply only to the 8-bit, or 16-bit, or 32-bit libraries, respectively, and are
-not available for libraries that have not been built. If an unavailable option
-is encountered, the "usage" information is output.
+apply only to the 8-bit, 16-bit, or 32-bit libraries, respectively, and are not
+available for libraries that have not been built. If an unavailable option is
+encountered, the "usage" information is output.
.
.
.SH OPTIONS
.rs
.TP 10
\fB--prefix\fP
-Writes the directory prefix used in the PCRE2 installation for architecture
-independent files (\fI/usr\fP on many systems, \fI/usr/local\fP on some
-systems) to the standard output.
+Writes the directory prefix used in the PCRE2 installation for
+architecture-independent files (\fI/usr\fP on many systems, \fI/usr/local\fP on
+some systems) to the standard output.
.TP 10
\fB--exec-prefix\fP
-Writes the directory prefix used in the PCRE2 installation for architecture
-dependent files (normally the same as \fB--prefix\fP) to the standard output.
+Writes the directory prefix used in the PCRE2 installation for
+architecture-dependent files (normally the same as \fB--prefix\fP) to the
+standard output.
.TP 10
\fB--version\fP
Writes the version number of the installed PCRE2 libraries to the standard
@@ -82,5 +83,5 @@ system. It has been subsequently revised as a generic PCRE2 man page.
.rs
.sp
.nf
-Last updated: 28 September 2014
+Last updated: 22 February 2025
.fi
diff --git a/doc/pcre2-config.txt b/doc/pcre2-config.txt
index 9c70b40..74592eb 100644
--- a/doc/pcre2-config.txt
+++ b/doc/pcre2-config.txt
@@ -16,51 +16,51 @@ DESCRIPTION
pcre2-config returns the configuration of the installed PCRE2 libraries
and the options required to compile a program to use them. Some of the
- options apply only to the 8-bit, or 16-bit, or 32-bit libraries, re-
- spectively, and are not available for libraries that have not been
- built. If an unavailable option is encountered, the "usage" information
- is output.
+ options apply only to the 8-bit, 16-bit, or 32-bit libraries, respec-
+ tively, and are not available for libraries that have not been built.
+ If an unavailable option is encountered, the "usage" information is
+ output.
OPTIONS
- --prefix Writes the directory prefix used in the PCRE2 installation
- for architecture independent files (/usr on many systems,
+ --prefix Writes the directory prefix used in the PCRE2 installation
+ for architecture-independent files (/usr on many systems,
/usr/local on some systems) to the standard output.
--exec-prefix
- Writes the directory prefix used in the PCRE2 installation
- for architecture dependent files (normally the same as --pre-
+ Writes the directory prefix used in the PCRE2 installation
+ for architecture-dependent files (normally the same as --pre-
fix) to the standard output.
--version Writes the version number of the installed PCRE2 libraries to
the standard output.
- --libs8 Writes to the standard output the command line options re-
- quired to link with the 8-bit PCRE2 library (-lpcre2-8 on
+ --libs8 Writes to the standard output the command line options re-
+ quired to link with the 8-bit PCRE2 library (-lpcre2-8 on
many systems).
- --libs16 Writes to the standard output the command line options re-
- quired to link with the 16-bit PCRE2 library (-lpcre2-16 on
+ --libs16 Writes to the standard output the command line options re-
+ quired to link with the 16-bit PCRE2 library (-lpcre2-16 on
many systems).
- --libs32 Writes to the standard output the command line options re-
- quired to link with the 32-bit PCRE2 library (-lpcre2-32 on
+ --libs32 Writes to the standard output the command line options re-
+ quired to link with the 32-bit PCRE2 library (-lpcre2-32 on
many systems).
--libs-posix
- Writes to the standard output the command line options re-
+ Writes to the standard output the command line options re-
quired to link with PCRE2's POSIX API wrapper library
(-lpcre2-posix -lpcre2-8 on many systems).
- --cflags Writes to the standard output the command line options re-
+ --cflags Writes to the standard output the command line options re-
quired to compile files that use PCRE2 (this may include some
-I options, but is blank on many systems).
--cflags-posix
- Writes to the standard output the command line options re-
- quired to compile files that use PCRE2's POSIX API wrapper
- library (this may include some -I options, but is blank on
+ Writes to the standard output the command line options re-
+ quired to compile files that use PCRE2's POSIX API wrapper
+ library (this may include some -I options, but is blank on
many systems).
@@ -71,14 +71,14 @@ SEE ALSO
AUTHOR
- This manual page was originally written by Mark Baker for the Debian
- GNU/Linux system. It has been subsequently revised as a generic PCRE2
+ This manual page was originally written by Mark Baker for the Debian
+ GNU/Linux system. It has been subsequently revised as a generic PCRE2
man page.
REVISION
- Last updated: 28 September 2014
+ Last updated: 22 February 2025
-PCRE2 10.46 28 September 2014 PCRE2-CONFIG(1)
+PCRE2 10.47 22 February 2025 PCRE2-CONFIG(1)
diff --git a/doc/pcre2.3 b/doc/pcre2.3
index 32c7c9c..1c3d9d2 100644
--- a/doc/pcre2.3
+++ b/doc/pcre2.3
@@ -1,4 +1,4 @@
-.TH PCRE2 3 "18 December 2024" "PCRE2 10.46"
+.TH PCRE2 3 "22 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH INTRODUCTION
@@ -16,14 +16,14 @@ obsolete and is no longer maintained.
.P
As well as Perl-style regular expression patterns, some features that appeared
in Python and the original PCRE before they appeared in Perl are available
-using the Python syntax. There is also some support for one or two .NET and
-Oniguruma syntax items, and there are options for requesting some minor changes
-that give better ECMAScript (aka JavaScript) compatibility.
+using the Python syntax. There is also support for some .NET and Oniguruma
+syntax items, and there are options for requesting minor changes that give
+better ECMAScript (JavaScript) compatibility.
.P
The source code for PCRE2 can be compiled to support strings of 8-bit, 16-bit,
or 32-bit code units, which means that up to three separate libraries may be
-installed, one for each code unit size. The size of code unit is not related to
-the bit size of the underlying hardware. In a 64-bit environment that also
+installed, one for each code unit size. The size of a code unit is not related
+to the bit size of the underlying hardware. In a 64-bit environment that also
supports 32-bit applications, versions of PCRE2 that are compiled in both
64-bit and 32-bit modes may be needed.
.P
@@ -87,7 +87,7 @@ found in the
and
.\" HTML
.\"
-\fBNON-AUTOTOOLS_BUILD\fP
+\fBNON-AUTOTOOLS-BUILD\fP
.\"
files in the source distribution.
.P
@@ -206,6 +206,6 @@ https://github.com/PCRE2Project/pcre2
.rs
.sp
.nf
-Last updated: 18 December 2024
+Last updated: 22 February 2025
Copyright (c) 1997-2021 University of Cambridge.
.fi
diff --git a/doc/pcre2.txt b/doc/pcre2.txt
index 9903edd..b0b564c 100644
--- a/doc/pcre2.txt
+++ b/doc/pcre2.txt
@@ -30,17 +30,17 @@ INTRODUCTION
As well as Perl-style regular expression patterns, some features that
appeared in Python and the original PCRE before they appeared in Perl
- are available using the Python syntax. There is also some support for
- one or two .NET and Oniguruma syntax items, and there are options for
- requesting some minor changes that give better ECMAScript (aka
- JavaScript) compatibility.
+ are available using the Python syntax. There is also support for some
+ .NET and Oniguruma syntax items, and there are options for requesting
+ minor changes that give better ECMAScript (JavaScript) compatibility.
- The source code for PCRE2 can be compiled to support strings of 8-bit,
+ The source code for PCRE2 can be compiled to support strings of 8-bit,
16-bit, or 32-bit code units, which means that up to three separate li-
- braries may be installed, one for each code unit size. The size of code
- unit is not related to the bit size of the underlying hardware. In a
- 64-bit environment that also supports 32-bit applications, versions of
- PCRE2 that are compiled in both 64-bit and 32-bit modes may be needed.
+ braries may be installed, one for each code unit size. The size of a
+ code unit is not related to the bit size of the underlying hardware. In
+ a 64-bit environment that also supports 32-bit applications, versions
+ of PCRE2 that are compiled in both 64-bit and 32-bit modes may be
+ needed.
The original work to extend PCRE to 16-bit and 32-bit code units was
done by Zoltan Herczeg and Christian Persch, respectively. In all three
@@ -76,7 +76,7 @@ INTRODUCTION
client to discover which features are available. The features them-
selves are described in the pcre2build page. Documentation about build-
ing PCRE2 for various operating systems can be found in the README and
- NON-AUTOTOOLS_BUILD files in the source distribution.
+ NON-AUTOTOOLS-BUILD files in the source distribution.
The libraries contains a number of undocumented internal functions and
data tables that are used by more than one of the exported external
@@ -185,11 +185,11 @@ AUTHORS
REVISION
- Last updated: 18 December 2024
+ Last updated: 22 February 2025
Copyright (c) 1997-2021 University of Cambridge.
-PCRE2 10.46 18 December 2024 PCRE2(3)
+PCRE2 10.47 22 February 2025 PCRE2(3)
------------------------------------------------------------------------------
@@ -381,7 +381,7 @@ PCRE2 NATIVE API STRING SUBSTITUTION FUNCTION
int pcre2_substitute(const pcre2_code *code, PCRE2_SPTR subject,
PCRE2_SIZE length, PCRE2_SIZE startoffset,
uint32_t options, pcre2_match_data *match_data,
- pcre2_match_context *mcontext, PCRE2_SPTR replacementz,
+ pcre2_match_context *mcontext, PCRE2_SPTR replacement,
PCRE2_SIZE rlength, PCRE2_UCHAR *outputbuffer,
PCRE2_SIZE *outlengthptr);
@@ -638,7 +638,7 @@ PCRE2 API OVERVIEW
Finally, there are functions for finding out information about a com-
piled pattern (pcre2_pattern_info()) and about the configuration with
- which PCRE2 was built (pcre2_config()).
+ which PCRE2 was built (pcre2_config()) and that it is using.
Functions with names ending with _free() are used for freeing memory
blocks of various sorts. In all cases, if one of these functions is
@@ -1337,61 +1337,69 @@ CHECKING BUILD-TIME OPTIONS
recursions, lookarounds, and atomic groups in pcre2_dfa_match(). Fur-
ther details are given with pcre2_set_depth_limit() above.
+ PCRE2_CONFIG_EFFECTIVE_LINKSIZE
+
+ The output is a uint32_t integer that contains the number of bytes the
+ library uses for internal linkage in compiled regular expressions. Its
+ value is derived from the value that was provided at build time and
+ that is described below by PCRE2_CONFIG_LINKSIZE.
+
PCRE2_CONFIG_HEAPLIMIT
- The output is a uint32_t integer that gives, in kibibytes, the default
- limit for the amount of heap memory used by pcre2_match() or
- pcre2_dfa_match(). Further details are given with
+ The output is a uint32_t integer that gives, in kibibytes, the default
+ limit for the amount of heap memory used by pcre2_match() or
+ pcre2_dfa_match(). Further details are given with
pcre2_set_heap_limit() above.
PCRE2_CONFIG_JIT
- The output is a uint32_t integer that is set to one if support for
- just-in-time compiling is included in the library; otherwise it is set
+ The output is a uint32_t integer that is set to one if support for
+ just-in-time compiling is included in the library; otherwise it is set
to zero. Note that having the support in the library does not guarantee
- that JIT will be used for any given match, and neither does it guaran-
- tee that JIT will actually be able to function, because it may not be
- able to allocate executable memory in some environments. There is a
+ that JIT will be used for any given match, and neither does it guaran-
+ tee that JIT will actually be able to function, because it may not be
+ able to allocate executable memory in some environments. There is a
special call to pcre2_jit_compile() that can be used to check this. See
the pcre2jit documentation for more details.
PCRE2_CONFIG_JITTARGET
- The where argument should point to a buffer that is at least 48 code
- units long. (The exact length required can be found by calling
- pcre2_config() with where set to NULL.) The buffer is filled with a
- string that contains the name of the architecture for which the JIT
- compiler is configured, for example "x86 32bit (little endian + un-
- aligned)". If JIT support is not available, PCRE2_ERROR_BADOPTION is
- returned, otherwise the number of code units used is returned. This is
+ The where argument should point to a buffer that is at least 64 code
+ units long. (The exact length required can be found by calling
+ pcre2_config() with where set to NULL.) The buffer is filled with a
+ string that contains the name of the architecture for which the JIT
+ compiler is configured, for example "x86 32bit (little endian + un-
+ aligned)". If JIT support is not available, PCRE2_ERROR_BADOPTION is
+ returned, otherwise the number of code units used is returned. This is
the length of the string, plus one unit for the terminating zero.
PCRE2_CONFIG_LINKSIZE
- The output is a uint32_t integer that contains the number of bytes used
- for internal linkage in compiled regular expressions. When PCRE2 is
- configured, the value can be set to 2, 3, or 4, with the default being
- 2. This is the value that is returned by pcre2_config(). However, when
- the 16-bit library is compiled, a value of 3 is rounded up to 4, and
- when the 32-bit library is compiled, internal linkages always use 4
- bytes, so the configured value is not relevant.
+ The output is a uint32_t integer that contains the number of bytes the
+ library was instructed to use for internal linkage in compiled regular
+ expressions. When PCRE2 is configured, the value can be set to 2, 3,
+ or 4, with the default being 2 for most libraries.
+
+ The actual number of bytes used depends on the size of the code units
+ that the library supports and can be higher. See PCRE2_CONFIG_EFFEC-
+ TIVE_LINKSIZE above for details.
The default value of 2 for the 8-bit and 16-bit libraries is sufficient
- for all but the most massive patterns, since it allows the size of the
- compiled pattern to be up to 65535 code units. Larger values allow
- larger regular expressions to be compiled by those two libraries, but
+ for all but the most massive patterns, since it allows the size of the
+ compiled pattern to be up to 65535 code units. Larger values allow
+ larger regular expressions to be compiled by those two libraries, but
at the expense of slower matching.
PCRE2_CONFIG_MATCHLIMIT
The output is a uint32_t integer that gives the default match limit for
- pcre2_match(). Further details are given with pcre2_set_match_limit()
+ pcre2_match(). Further details are given with pcre2_set_match_limit()
above.
PCRE2_CONFIG_NEWLINE
- The output is a uint32_t integer whose value specifies the default
- character sequence that is recognized as meaning "newline". The values
+ The output is a uint32_t integer whose value specifies the default
+ character sequence that is recognized as meaning "newline". The values
are:
PCRE2_NEWLINE_CR Carriage return (CR)
@@ -1401,23 +1409,23 @@ CHECKING BUILD-TIME OPTIONS
PCRE2_NEWLINE_ANYCRLF Any of CR, LF, or CRLF
PCRE2_NEWLINE_NUL The NUL character (binary zero)
- The default should normally correspond to the standard sequence for
+ The default should normally correspond to the standard sequence for
your operating system.
PCRE2_CONFIG_NEVER_BACKSLASH_C
- The output is a uint32_t integer that is set to one if the use of \C
- was permanently disabled when PCRE2 was built; otherwise it is set to
+ The output is a uint32_t integer that is set to one if the use of \C
+ was permanently disabled when PCRE2 was built; otherwise it is set to
zero.
PCRE2_CONFIG_PARENSLIMIT
- The output is a uint32_t integer that gives the maximum depth of nest-
+ The output is a uint32_t integer that gives the maximum depth of nest-
ing of parentheses (of any kind) in a pattern. This limit is imposed to
- cap the amount of system stack used when a pattern is compiled. It is
- specified when PCRE2 is built; the default is 250. This limit does not
+ cap the amount of system stack used when a pattern is compiled. It is
+ specified when PCRE2 is built; the default is 250. This limit does not
take into account the stack that may already be used by the calling ap-
- plication. For finer control over compilation stack usage, see
+ plication. For finer control over compilation stack usage, see
pcre2_set_compile_recursion_guard().
PCRE2_CONFIG_STACKRECURSE
@@ -1428,30 +1436,30 @@ CHECKING BUILD-TIME OPTIONS
PCRE2_CONFIG_TABLES_LENGTH
The output is a uint32_t integer that gives the length of PCRE2's char-
- acter processing tables in bytes. For details of these tables see the
+ acter processing tables in bytes. For details of these tables see the
section on locale support below.
PCRE2_CONFIG_UNICODE_VERSION
- The where argument should point to a buffer that is at least 24 code
- units long. (The exact length required can be found by calling
- pcre2_config() with where set to NULL.) If PCRE2 has been compiled
- without Unicode support, the buffer is filled with the text "Unicode
- not supported". Otherwise, the Unicode version string (for example,
- "8.0.0") is inserted. The number of code units used is returned. This
+ The where argument should point to a buffer that is at least 24 code
+ units long. (The exact length required can be found by calling
+ pcre2_config() with where set to NULL.) If PCRE2 has been compiled
+ without Unicode support, the buffer is filled with the text "Unicode
+ not supported". Otherwise, the Unicode version string (for example,
+ "8.0.0") is inserted. The number of code units used is returned. This
is the length of the string plus one unit for the terminating zero.
PCRE2_CONFIG_UNICODE
- The output is a uint32_t integer that is set to one if Unicode support
- is available; otherwise it is set to zero. Unicode support implies UTF
+ The output is a uint32_t integer that is set to one if Unicode support
+ is available; otherwise it is set to zero. Unicode support implies UTF
support.
PCRE2_CONFIG_VERSION
- The where argument should point to a buffer that is at least 24 code
- units long. (The exact length required can be found by calling
- pcre2_config() with where set to NULL.) The buffer is filled with the
+ The where argument should point to a buffer that is at least 24 code
+ units long. (The exact length required can be found by calling
+ pcre2_config() with where set to NULL.) The buffer is filled with the
PCRE2 version string, zero-terminated. The number of code units used is
returned. This is the length of the string plus one unit for the termi-
nating zero.
@@ -1469,107 +1477,107 @@ COMPILING A PATTERN
pcre2_code *pcre2_code_copy_with_tables(const pcre2_code *code);
- The pcre2_compile() function compiles a pattern into an internal form.
- The pattern is defined by a pointer to a string of code units and a
+ The pcre2_compile() function compiles a pattern into an internal form.
+ The pattern is defined by a pointer to a string of code units and a
length in code units. If the pattern is zero-terminated, the length can
- be specified as PCRE2_ZERO_TERMINATED. A NULL pattern pointer with a
- length of zero is treated as an empty string (NULL with a non-zero
- length causes an error return). The function returns a pointer to a
+ be specified as PCRE2_ZERO_TERMINATED. A NULL pattern pointer with a
+ length of zero is treated as an empty string (NULL with a non-zero
+ length causes an error return). The function returns a pointer to a
block of memory that contains the compiled pattern and related data, or
NULL if an error occurred.
- If the compile context argument ccontext is NULL, memory for the com-
- piled pattern is obtained by calling malloc(). Otherwise, it is ob-
+ If the compile context argument ccontext is NULL, memory for the com-
+ piled pattern is obtained by calling malloc(). Otherwise, it is ob-
tained from the same memory function that was used for the compile con-
text. The caller must free the memory by calling pcre2_code_free() when
it is no longer needed. If pcre2_code_free() is called with a NULL ar-
gument, it returns immediately, without doing anything.
The function pcre2_code_copy() makes a copy of the compiled code in new
- memory, using the same memory allocator as was used for the original.
- However, if the code has been processed by the JIT compiler (see be-
- low), the JIT information cannot be copied (because it is position-de-
- pendent). The new copy can initially be used only for non-JIT match-
- ing, though it can be passed to pcre2_jit_compile() if required. If
+ memory, using the same memory allocator as was used for the original.
+ However, if the code has been processed by the JIT compiler (see be-
+ low), the JIT information cannot be copied (because it is position-de-
+ pendent). The new copy can initially be used only for non-JIT match-
+ ing, though it can be passed to pcre2_jit_compile() if required. If
pcre2_code_copy() is called with a NULL argument, it returns NULL.
The pcre2_code_copy() function provides a way for individual threads in
- a multithreaded application to acquire a private copy of shared com-
- piled code. However, it does not make a copy of the character tables
- used by the compiled pattern; the new pattern code points to the same
- tables as the original code. (See "Locale Support" below for details
- of these character tables.) In many applications the same tables are
- used throughout, so this behaviour is appropriate. Nevertheless, there
+ a multithreaded application to acquire a private copy of shared com-
+ piled code. However, it does not make a copy of the character tables
+ used by the compiled pattern; the new pattern code points to the same
+ tables as the original code. (See "Locale Support" below for details
+ of these character tables.) In many applications the same tables are
+ used throughout, so this behaviour is appropriate. Nevertheless, there
are occasions when a copy of a compiled pattern and the relevant tables
- are needed. The pcre2_code_copy_with_tables() provides this facility.
- Copies of both the code and the tables are made, with the new code
- pointing to the new tables. The memory for the new tables is automati-
- cally freed when pcre2_code_free() is called for the new copy of the
- compiled code. If pcre2_code_copy_with_tables() is called with a NULL
+ are needed. The pcre2_code_copy_with_tables() provides this facility.
+ Copies of both the code and the tables are made, with the new code
+ pointing to the new tables. The memory for the new tables is automati-
+ cally freed when pcre2_code_free() is called for the new copy of the
+ compiled code. If pcre2_code_copy_with_tables() is called with a NULL
argument, it returns NULL.
- NOTE: When one of the matching functions is called, pointers to the
+ NOTE: When one of the matching functions is called, pointers to the
compiled pattern and the subject string are set in the match data block
- so that they can be referenced by the substring extraction functions
- after a successful match. After running a match, you must not free a
- compiled pattern or a subject string until after all operations on the
- match data block have taken place, unless, in the case of the subject
- string, you have used the PCRE2_COPY_MATCHED_SUBJECT option, which is
- described in the section entitled "Option bits for pcre2_match()" be-
+ so that they can be referenced by the substring extraction functions
+ after a successful match. After running a match, you must not free a
+ compiled pattern or a subject string until after all operations on the
+ match data block have taken place, unless, in the case of the subject
+ string, you have used the PCRE2_COPY_MATCHED_SUBJECT option, which is
+ described in the section entitled "Option bits for pcre2_match()" be-
low.
- The options argument for pcre2_compile() contains various bit settings
- that affect the compilation. It should be zero if none of them are re-
- quired. The available options are described below. Some of them (in
- particular, those that are compatible with Perl, but some others as
- well) can also be set and unset from within the pattern (see the de-
+ The options argument for pcre2_compile() contains various bit settings
+ that affect the compilation. It should be zero if none of them are re-
+ quired. The available options are described below. Some of them (in
+ particular, those that are compatible with Perl, but some others as
+ well) can also be set and unset from within the pattern (see the de-
tailed description in the pcre2pattern documentation).
- For those options that can be different in different parts of the pat-
- tern, the contents of the options argument specifies their settings at
- the start of compilation. The PCRE2_ANCHORED, PCRE2_ENDANCHORED, and
- PCRE2_NO_UTF_CHECK options can be set at the time of matching as well
+ For those options that can be different in different parts of the pat-
+ tern, the contents of the options argument specifies their settings at
+ the start of compilation. The PCRE2_ANCHORED, PCRE2_ENDANCHORED, and
+ PCRE2_NO_UTF_CHECK options can be set at the time of matching as well
as at compile time.
Some additional options and less frequently required compile-time para-
- meters (for example, the newline setting) can be provided in a compile
+ meters (for example, the newline setting) can be provided in a compile
context (as described above).
If errorcode or erroroffset is NULL, pcre2_compile() returns NULL imme-
- diately. Otherwise, the variables to which these point are set to an
+ diately. Otherwise, the variables to which these point are set to an
error code and an offset (number of code units) within the pattern, re-
spectively, when pcre2_compile() returns NULL because a compilation er-
ror has occurred.
There are over 100 positive error codes that pcre2_compile() may return
if it finds an error in the pattern. There are also some negative error
- codes that are used for invalid UTF strings when validity checking is
- in force. These are the same as given by pcre2_match() and
+ codes that are used for invalid UTF strings when validity checking is
+ in force. These are the same as given by pcre2_match() and
pcre2_dfa_match(), and are described in the pcre2unicode documentation.
- There is no separate documentation for the positive error codes, be-
- cause the textual error messages that are obtained by calling the
+ There is no separate documentation for the positive error codes, be-
+ cause the textual error messages that are obtained by calling the
pcre2_get_error_message() function (see "Obtaining a textual error mes-
- sage" below) should be self-explanatory. Macro names starting with
- PCRE2_ERROR_ are defined for both positive and negative error codes in
- pcre2.h. When compilation is successful errorcode is set to a value
- that returns the message "no error" if passed to pcre2_get_error_mes-
+ sage" below) should be self-explanatory. Macro names starting with
+ PCRE2_ERROR_ are defined for both positive and negative error codes in
+ pcre2.h. When compilation is successful errorcode is set to a value
+ that returns the message "no error" if passed to pcre2_get_error_mes-
sage().
The value returned in erroroffset is an indication of where in the pat-
- tern an error occurred. When there is no error, zero is returned. A
- non-zero value is not necessarily the furthest point in the pattern
- that was read. For example, after the error "lookbehind assertion is
- not fixed length", the error offset points to the start of the failing
+ tern an error occurred. When there is no error, zero is returned. A
+ non-zero value is not necessarily the furthest point in the pattern
+ that was read. For example, after the error "lookbehind assertion is
+ not fixed length", the error offset points to the start of the failing
assertion. For an invalid UTF-8 or UTF-16 string, the offset is that of
the first code unit of the failing character.
- Some errors are not detected until the whole pattern has been scanned;
- in these cases, the offset passed back is the length of the pattern.
- Note that the offset is in code units, not characters, even in a UTF
+ Some errors are not detected until the whole pattern has been scanned;
+ in these cases, the offset passed back is the length of the pattern.
+ Note that the offset is in code units, not characters, even in a UTF
mode. It may sometimes point into the middle of a UTF-8 or UTF-16 char-
acter.
- This code fragment shows a typical straightforward call to pcre2_com-
+ This code fragment shows a typical straightforward call to pcre2_com-
pile():
pcre2_code *re;
@@ -1586,81 +1594,80 @@ COMPILING A PATTERN
Main compile options
- The following names for option bits are defined in the pcre2.h header
+ The following names for option bits are defined in the pcre2.h header
file:
PCRE2_ANCHORED
If this bit is set, the pattern is forced to be "anchored", that is, it
- is constrained to match only at the first matching point in the string
- that is being searched (the "subject string"). This effect can also be
- achieved by appropriate constructs in the pattern itself, which is the
+ is constrained to match only at the first matching point in the string
+ that is being searched (the "subject string"). This effect can also be
+ achieved by appropriate constructs in the pattern itself, which is the
only way to do it in Perl.
PCRE2_ALLOW_EMPTY_CLASS
- By default, for compatibility with Perl, a closing square bracket that
- immediately follows an opening one is treated as a data character for
- the class. When PCRE2_ALLOW_EMPTY_CLASS is set, it terminates the
+ By default, for compatibility with Perl, a closing square bracket that
+ immediately follows an opening one is treated as a data character for
+ the class. When PCRE2_ALLOW_EMPTY_CLASS is set, it terminates the
class, which therefore contains no characters and so can never match.
PCRE2_ALT_BSUX
- This option request alternative handling of three escape sequences,
- which makes PCRE2's behaviour more like ECMAscript (aka JavaScript).
+ This option request alternative handling of three escape sequences,
+ which makes PCRE2's behaviour more like ECMAscript (aka JavaScript).
When it is set:
(1) \U matches an upper case "U" character; by default \U causes a com-
pile time error (Perl uses \U to upper case subsequent characters).
(2) \u matches a lower case "u" character unless it is followed by four
- hexadecimal digits, in which case the hexadecimal number defines the
- code point to match. By default, \u causes a compile time error (Perl
+ hexadecimal digits, in which case the hexadecimal number defines the
+ code point to match. By default, \u causes a compile time error (Perl
uses it to upper case the following character).
- (3) \x matches a lower case "x" character unless it is followed by two
- hexadecimal digits, in which case the hexadecimal number defines the
- code point to match. By default, as in Perl, a hexadecimal number is
- always expected after \x, but it may have zero, one, or two digits (so,
- for example, \xz matches a binary zero character followed by z).
+ (3) \x matches a lower case "x" character unless it is followed by two
+ hexadecimal digits, in which case the hexadecimal number defines the
+ code point to match. By default, as in Perl, a hexadecimal number is
+ always expected after \x, but it may have one or two digits.
ECMAscript 6 added additional functionality to \u. This can be accessed
- using the PCRE2_EXTRA_ALT_BSUX extra option (see "Extra compile op-
+ using the PCRE2_EXTRA_ALT_BSUX extra option (see "Extra compile op-
tions" below). Note that this alternative escape handling applies only
- to patterns. Neither of these options affects the processing of re-
+ to patterns. Neither of these options affects the processing of re-
placement strings passed to pcre2_substitute().
PCRE2_ALT_CIRCUMFLEX
In multiline mode (when PCRE2_MULTILINE is set), the circumflex
- metacharacter matches at the start of the subject (unless PCRE2_NOTBOL
- is set), and also after any internal newline. However, it does not
+ metacharacter matches at the start of the subject (unless PCRE2_NOTBOL
+ is set), and also after any internal newline. However, it does not
match after a newline at the end of the subject, for compatibility with
- Perl. If you want a multiline circumflex also to match after a termi-
+ Perl. If you want a multiline circumflex also to match after a termi-
nating newline, you must set PCRE2_ALT_CIRCUMFLEX.
PCRE2_ALT_EXTENDED_CLASS
- Alters the parsing of character classes to follow the extended syntax
+ Alters the parsing of character classes to follow the extended syntax
described by Unicode UTS#18. The PCRE2_ALT_EXTENDED_CLASS option has no
- impact on the behaviour of the Perl-specific "(?[...])" syntax for ex-
- tended classes, but instead enables the alternative syntax of extended
- class behaviour inside ordinary "[...]" character classes. See the
- pcre2pattern documentation for details of the character classes sup-
+ impact on the behaviour of the Perl-specific "(?[...])" syntax for ex-
+ tended classes, but instead enables the alternative syntax of extended
+ class behaviour inside ordinary "[...]" character classes. See the
+ pcre2pattern documentation for details of the character classes sup-
ported.
PCRE2_ALT_VERBNAMES
- By default, for compatibility with Perl, the name in any verb sequence
- such as (*MARK:NAME) is any sequence of characters that does not in-
- clude a closing parenthesis. The name is not processed in any way, and
- it is not possible to include a closing parenthesis in the name. How-
- ever, if the PCRE2_ALT_VERBNAMES option is set, normal backslash pro-
- cessing is applied to verb names and only an unescaped closing paren-
- thesis terminates the name. A closing parenthesis can be included in a
- name either as \) or between \Q and \E. If the PCRE2_EXTENDED or
- PCRE2_EXTENDED_MORE option is set with PCRE2_ALT_VERBNAMES, unescaped
- whitespace in verb names is skipped and #-comments are recognized, ex-
+ By default, for compatibility with Perl, the name in any verb sequence
+ such as (*MARK:NAME) is any sequence of characters that does not in-
+ clude a closing parenthesis. The name is not processed in any way, and
+ it is not possible to include a closing parenthesis in the name. How-
+ ever, if the PCRE2_ALT_VERBNAMES option is set, normal backslash pro-
+ cessing is applied to verb names and only an unescaped closing paren-
+ thesis terminates the name. A closing parenthesis can be included in a
+ name either as \) or between \Q and \E. If the PCRE2_EXTENDED or
+ PCRE2_EXTENDED_MORE option is set with PCRE2_ALT_VERBNAMES, unescaped
+ white space in verb names is skipped and #-comments are recognized, ex-
actly as in the rest of the pattern.
PCRE2_AUTO_CALLOUT
@@ -2901,15 +2908,9 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION
starting point to discover that it is preceded by a letter.
Finding all the matches in a subject is tricky when the pattern can
- match an empty string. It is possible to emulate Perl's /g behaviour by
- first trying the match again at the same offset, with the
- PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED options, and then if that
- fails, advancing the starting offset and trying an ordinary match
- again. There is some code that demonstrates how to do this in the
- pcre2demo sample program. In the most general case, you have to check
- to see if the newline convention recognizes CRLF as a newline, and if
- so, and the current character is CR followed by LF, advance the start-
- ing offset by two characters instead of one.
+ match an empty string. PCRE2 includes a helper API to assist with this;
+ see the section entitled "Iterating over all matches" below for de-
+ tails.
If a non-zero starting offset is passed when the pattern is anchored, a
single attempt to match at the given offset is made. This can only suc-
@@ -2939,156 +2940,156 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION
The PCRE2_ANCHORED option limits pcre2_match() to matching at the first
matching position. If a pattern was compiled with PCRE2_ANCHORED, or
turned out to be anchored by virtue of its contents, it cannot be made
- unachored at matching time. Note that setting the option at match time
+ unanchored at matching time. Note that setting the option at match time
disables JIT matching.
PCRE2_COPY_MATCHED_SUBJECT
- By default, a pointer to the subject is remembered in the match data
- block so that, after a successful match, it can be referenced by the
- substring extraction functions. This means that the subject's memory
- must not be freed until all such operations are complete. For some ap-
- plications where the lifetime of the subject string is not guaranteed,
- it may be necessary to make a copy of the subject string, but it is
- wasteful to do this unless the match is successful. After a successful
- match, if PCRE2_COPY_MATCHED_SUBJECT is set, the subject is copied and
- the new pointer is remembered in the match data block instead of the
- original subject pointer. The memory allocator that was used for the
- match block itself is used. The copy is automatically freed when
- pcre2_match_data_free() is called to free the match data block. It is
+ By default, a pointer to the subject is remembered in the match data
+ block so that, after a successful match, it can be referenced by the
+ substring extraction functions. This means that the subject's memory
+ must not be freed until all such operations are complete. For some ap-
+ plications where the lifetime of the subject string is not guaranteed,
+ it may be necessary to make a copy of the subject string, but it is
+ wasteful to do this unless the match is successful. After a successful
+ match, if PCRE2_COPY_MATCHED_SUBJECT is set, the subject is copied and
+ the new pointer is remembered in the match data block instead of the
+ original subject pointer. The memory allocator that was used for the
+ match block itself is used. The copy is automatically freed when
+ pcre2_match_data_free() is called to free the match data block. It is
also automatically freed if the match data block is re-used for another
match operation.
PCRE2_DISABLE_RECURSELOOP_CHECK
- This option is relevant only to pcre2_match() for interpretive match-
- ing. It is ignored when JIT is used, and is forbidden for
+ This option is relevant only to pcre2_match() for interpretive match-
+ ing. It is ignored when JIT is used, and is forbidden for
pcre2_dfa_match().
The use of recursion in patterns can lead to infinite loops. In the in-
- terpretive matcher these would be eventually caught by the match or
+ terpretive matcher these would be eventually caught by the match or
heap limits, but this could take a long time and/or use a lot of memory
- if the limits are large. There is therefore a check at the start of
- each recursion. If the same group is still active from a previous
- call, and the current subject pointer is the same as it was at the
- start of that group, and the furthest inspected character of the sub-
+ if the limits are large. There is therefore a check at the start of
+ each recursion. If the same group is still active from a previous
+ call, and the current subject pointer is the same as it was at the
+ start of that group, and the furthest inspected character of the sub-
ject has not changed, an error is generated.
- There are rare cases of matches that would complete, but nevertheless
- trigger this error. This option disables the check. It is provided
+ There are rare cases of matches that would complete, but nevertheless
+ trigger this error. This option disables the check. It is provided
mainly for testing when comparing JIT and interpretive behaviour.
PCRE2_ENDANCHORED
- If the PCRE2_ENDANCHORED option is set, any string that pcre2_match()
- matches must be right at the end of the subject string. Note that set-
+ If the PCRE2_ENDANCHORED option is set, any string that pcre2_match()
+ matches must be right at the end of the subject string. Note that set-
ting the option at match time disables JIT matching.
PCRE2_NOTBOL
This option specifies that first character of the subject string is not
- the beginning of a line, so the circumflex metacharacter should not
- match before it. Setting this without having set PCRE2_MULTILINE at
+ the beginning of a line, so the circumflex metacharacter should not
+ match before it. Setting this without having set PCRE2_MULTILINE at
compile time causes circumflex never to match. This option affects only
the behaviour of the circumflex metacharacter. It does not affect \A.
PCRE2_NOTEOL
This option specifies that the end of the subject string is not the end
- of a line, so the dollar metacharacter should not match it nor (except
- in multiline mode) a newline immediately before it. Setting this with-
- out having set PCRE2_MULTILINE at compile time causes dollar never to
+ of a line, so the dollar metacharacter should not match it nor (except
+ in multiline mode) a newline immediately before it. Setting this with-
+ out having set PCRE2_MULTILINE at compile time causes dollar never to
match. This option affects only the behaviour of the dollar metacharac-
ter. It does not affect \Z or \z.
PCRE2_NOTEMPTY
An empty string is not considered to be a valid match if this option is
- set. If there are alternatives in the pattern, they are tried. If all
- the alternatives match the empty string, the entire match fails. For
+ set. If there are alternatives in the pattern, they are tried. If all
+ the alternatives match the empty string, the entire match fails. For
example, if the pattern
a?b?
- is applied to a string not beginning with "a" or "b", it matches an
+ is applied to a string not beginning with "a" or "b", it matches an
empty string at the start of the subject. With PCRE2_NOTEMPTY set, this
- match is not valid, so pcre2_match() searches further into the string
+ match is not valid, so pcre2_match() searches further into the string
for occurrences of "a" or "b".
PCRE2_NOTEMPTY_ATSTART
- This is like PCRE2_NOTEMPTY, except that it locks out an empty string
+ This is like PCRE2_NOTEMPTY, except that it locks out an empty string
match only at the first matching position, that is, at the start of the
- subject plus the starting offset. An empty string match later in the
+ subject plus the starting offset. An empty string match later in the
subject is permitted. If the pattern is anchored, such a match can oc-
cur only if the pattern contains \K.
PCRE2_NO_JIT
- By default, if a pattern has been successfully processed by
- pcre2_jit_compile(), JIT is automatically used when pcre2_match() is
- called with options that JIT supports. Setting PCRE2_NO_JIT disables
+ By default, if a pattern has been successfully processed by
+ pcre2_jit_compile(), JIT is automatically used when pcre2_match() is
+ called with options that JIT supports. Setting PCRE2_NO_JIT disables
the use of JIT; it forces matching to be done by the interpreter.
PCRE2_NO_UTF_CHECK
When PCRE2_UTF is set at compile time, the validity of the subject as a
- UTF string is checked unless PCRE2_NO_UTF_CHECK is passed to
+ UTF string is checked unless PCRE2_NO_UTF_CHECK is passed to
pcre2_match() or PCRE2_MATCH_INVALID_UTF was passed to pcre2_compile().
The latter special case is discussed in detail in the pcre2unicode doc-
umentation.
- In the default case, if a non-zero starting offset is given, the check
- is applied only to that part of the subject that could be inspected
- during matching, and there is a check that the starting offset points
- to the first code unit of a character or to the end of the subject. If
- there are no lookbehind assertions in the pattern, the check starts at
+ In the default case, if a non-zero starting offset is given, the check
+ is applied only to that part of the subject that could be inspected
+ during matching, and there is a check that the starting offset points
+ to the first code unit of a character or to the end of the subject. If
+ there are no lookbehind assertions in the pattern, the check starts at
the starting offset. Otherwise, it starts at the length of the longest
- lookbehind before the starting offset, or at the start of the subject
- if there are not that many characters before the starting offset. Note
+ lookbehind before the starting offset, or at the start of the subject
+ if there are not that many characters before the starting offset. Note
that the sequences \b and \B are one-character lookbehinds.
The check is carried out before any other processing takes place, and a
- negative error code is returned if the check fails. There are several
- UTF error codes for each code unit width, corresponding to different
- problems with the code unit sequence. There are discussions about the
- validity of UTF-8 strings, UTF-16 strings, and UTF-32 strings in the
+ negative error code is returned if the check fails. There are several
+ UTF error codes for each code unit width, corresponding to different
+ problems with the code unit sequence. There are discussions about the
+ validity of UTF-8 strings, UTF-16 strings, and UTF-32 strings in the
pcre2unicode documentation.
If you know that your subject is valid, and you want to skip this check
for performance reasons, you can set the PCRE2_NO_UTF_CHECK option when
- calling pcre2_match(). You might want to do this for the second and
- subsequent calls to pcre2_match() if you are making repeated calls to
+ calling pcre2_match(). You might want to do this for the second and
+ subsequent calls to pcre2_match() if you are making repeated calls to
find multiple matches in the same subject string.
- Warning: Unless PCRE2_MATCH_INVALID_UTF was set at compile time, when
- PCRE2_NO_UTF_CHECK is set at match time the effect of passing an in-
+ Warning: Unless PCRE2_MATCH_INVALID_UTF was set at compile time, when
+ PCRE2_NO_UTF_CHECK is set at match time the effect of passing an in-
valid string as a subject, or an invalid value of startoffset, is unde-
- fined. Your program may crash or loop indefinitely or give wrong re-
+ fined. Your program may crash or loop indefinitely or give wrong re-
sults.
PCRE2_PARTIAL_HARD
PCRE2_PARTIAL_SOFT
These options turn on the partial matching feature. A partial match oc-
- curs if the end of the subject string is reached successfully, but
+ curs if the end of the subject string is reached successfully, but
there are not enough subject characters to complete the match. In addi-
- tion, either at least one character must have been inspected or the
- pattern must contain a lookbehind, or the pattern must be one that
+ tion, either at least one character must have been inspected or the
+ pattern must contain a lookbehind, or the pattern must be one that
could match an empty string.
- If this situation arises when PCRE2_PARTIAL_SOFT (but not PCRE2_PAR-
+ If this situation arises when PCRE2_PARTIAL_SOFT (but not PCRE2_PAR-
TIAL_HARD) is set, matching continues by testing any remaining alterna-
- tives. Only if no complete match can be found is PCRE2_ERROR_PARTIAL
- returned instead of PCRE2_ERROR_NOMATCH. In other words, PCRE2_PAR-
- TIAL_SOFT specifies that the caller is prepared to handle a partial
+ tives. Only if no complete match can be found is PCRE2_ERROR_PARTIAL
+ returned instead of PCRE2_ERROR_NOMATCH. In other words, PCRE2_PAR-
+ TIAL_SOFT specifies that the caller is prepared to handle a partial
match, but only if no complete match can be found.
- If PCRE2_PARTIAL_HARD is set, it overrides PCRE2_PARTIAL_SOFT. In this
- case, if a partial match is found, pcre2_match() immediately returns
- PCRE2_ERROR_PARTIAL, without considering any other alternatives. In
+ If PCRE2_PARTIAL_HARD is set, it overrides PCRE2_PARTIAL_SOFT. In this
+ case, if a partial match is found, pcre2_match() immediately returns
+ PCRE2_ERROR_PARTIAL, without considering any other alternatives. In
other words, when PCRE2_PARTIAL_HARD is set, a partial match is consid-
- ered to be more important that an alternative complete match.
+ ered to be more important than an alternative complete match.
There is a more detailed discussion of partial and multi-segment match-
ing, with examples, in the pcre2partial documentation.
@@ -3096,38 +3097,38 @@ MATCHING A PATTERN: THE TRADITIONAL FUNCTION
NEWLINE HANDLING WHEN MATCHING
- When PCRE2 is built, a default newline convention is set; this is usu-
- ally the standard convention for the operating system. The default can
- be overridden in a compile context by calling pcre2_set_newline(). It
- can also be overridden by starting a pattern string with, for example,
- (*CRLF), as described in the section on newline conventions in the
- pcre2pattern page. During matching, the newline choice affects the be-
- haviour of the dot, circumflex, and dollar metacharacters. It may also
- alter the way the match starting position is advanced after a match
+ When PCRE2 is built, a default newline convention is set; this is usu-
+ ally the standard convention for the operating system. The default can
+ be overridden in a compile context by calling pcre2_set_newline(). It
+ can also be overridden by starting a pattern string with, for example,
+ (*CRLF), as described in the section on newline conventions in the
+ pcre2pattern page. During matching, the newline choice affects the be-
+ haviour of the dot, circumflex, and dollar metacharacters. It may also
+ alter the way the match starting position is advanced after a match
failure for an unanchored pattern.
When PCRE2_NEWLINE_CRLF, PCRE2_NEWLINE_ANYCRLF, or PCRE2_NEWLINE_ANY is
- set as the newline convention, and a match attempt for an unanchored
+ set as the newline convention, and a match attempt for an unanchored
pattern fails when the current starting position is at a CRLF sequence,
- and the pattern contains no explicit matches for CR or LF characters,
- the match position is advanced by two characters instead of one, in
+ and the pattern contains no explicit matches for CR or LF characters,
+ the match position is advanced by two characters instead of one, in
other words, to after the CRLF.
The above rule is a compromise that makes the most common cases work as
- expected. For example, if the pattern is .+A (and the PCRE2_DOTALL op-
- tion is not set), it does not match the string "\r\nA" because, after
- failing at the start, it skips both the CR and the LF before retrying.
- However, the pattern [\r\n]A does match that string, because it con-
+ expected. For example, if the pattern is .+A (and the PCRE2_DOTALL op-
+ tion is not set), it does not match the string "\r\nA" because, after
+ failing at the start, it skips both the CR and the LF before retrying.
+ However, the pattern [\r\n]A does match that string, because it con-
tains an explicit CR or LF reference, and so advances only by one char-
acter after the first failure.
An explicit match for CR of LF is either a literal appearance of one of
- those characters in the pattern, or one of the \r or \n or equivalent
+ those characters in the pattern, or one of the \r or \n or equivalent
octal or hexadecimal escape sequences. Implicit matches such as [^X] do
- not count, nor does \s, even though it includes CR and LF in the char-
+ not count, nor does \s, even though it includes CR and LF in the char-
acters that it matches.
- Notwithstanding the above, anomalous effects may still occur when CRLF
+ Notwithstanding the above, anomalous effects may still occur when CRLF
is a valid newline sequence and explicit \r or \n escapes appear in the
pattern.
@@ -3138,51 +3139,52 @@ HOW PCRE2_MATCH() RETURNS A STRING AND CAPTURED SUBSTRINGS
PCRE2_SIZE *pcre2_get_ovector_pointer(pcre2_match_data *match_data);
- In general, a pattern matches a certain portion of the subject, and in
- addition, further substrings from the subject may be picked out by
- parenthesized parts of the pattern. Following the usage in Jeffrey
- Friedl's book, this is called "capturing" in what follows, and the
- phrase "capture group" (Perl terminology) is used for a fragment of a
- pattern that picks out a substring. PCRE2 supports several other kinds
+ In general, a pattern matches a certain portion of the subject, and in
+ addition, further substrings from the subject may be picked out by
+ parenthesized parts of the pattern. Following the usage in Jeffrey
+ Friedl's book, this is called "capturing" in what follows, and the
+ phrase "capture group" (Perl terminology) is used for a fragment of a
+ pattern that picks out a substring. PCRE2 supports several other kinds
of parenthesized group that do not cause substrings to be captured. The
- pcre2_pattern_info() function can be used to find out how many capture
+ pcre2_pattern_info() function can be used to find out how many capture
groups there are in a compiled pattern.
- You can use auxiliary functions for accessing captured substrings by
+ You can use auxiliary functions for accessing captured substrings by
number or by name, as described in sections below.
Alternatively, you can make direct use of the vector of PCRE2_SIZE val-
- ues, called the ovector, which contains the offsets of captured
- strings. It is part of the match data block. The function
- pcre2_get_ovector_pointer() returns the address of the ovector, and
+ ues, called the ovector, which contains the offsets of captured
+ strings. It is part of the match data block. The function
+ pcre2_get_ovector_pointer() returns the address of the ovector, and
pcre2_get_ovector_count() returns the number of pairs of values it con-
tains.
Within the ovector, the first in each pair of values is set to the off-
set of the first code unit of a substring, and the second is set to the
- offset of the first code unit after the end of a substring. These val-
- ues are always code unit offsets, not character offsets. That is, they
+ offset of the first code unit after the end of a substring. These val-
+ ues are always code unit offsets, not character offsets. That is, they
are byte offsets in the 8-bit library, 16-bit offsets in the 16-bit li-
brary, and 32-bit offsets in the 32-bit library.
- After a partial match (error return PCRE2_ERROR_PARTIAL), only the
- first pair of offsets (that is, ovector[0] and ovector[1]) are set.
- They identify the part of the subject that was partially matched. See
+ After a partial match (error return PCRE2_ERROR_PARTIAL), only the
+ first pair of offsets (that is, ovector[0] and ovector[1]) are set.
+ They identify the part of the subject that was partially matched. See
the pcre2partial documentation for details of partial matching.
- After a fully successful match, the first pair of offsets identifies
- the portion of the subject string that was matched by the entire pat-
- tern. The next pair is used for the first captured substring, and so
- on. The value returned by pcre2_match() is one more than the highest
- numbered pair that has been set. For example, if two substrings have
- been captured, the returned value is 3. If there are no captured sub-
+ After a fully successful match, the first pair of offsets identifies
+ the portion of the subject string that was matched by the entire pat-
+ tern. The next pair is used for the first captured substring, and so
+ on. The value returned by pcre2_match() is one more than the highest
+ numbered pair that has been set. For example, if two substrings have
+ been captured, the returned value is 3. If there are no captured sub-
strings, the return value from a successful match is 1, indicating that
just the first pair of offsets has been set.
- If a pattern uses the \K escape sequence within a positive assertion,
- the reported start of a successful match can be greater than the end of
- the match. For example, if the pattern (?=ab\K) is matched against
- "ab", the start and end offset values for the match are 2 and 0.
+ If a pattern uses the \K escape sequence within a positive lookahead
+ assertion, the reported start of a successful match can be greater than
+ the end of the match. For example, if the pattern (?=ab\K) is matched
+ against "ab", the start and end offset values for the match are 2 and
+ 0.
If a capture group is matched repeatedly within a single match opera-
tion, it is the last portion of the subject that it matched that is re-
@@ -3398,8 +3400,92 @@ OBTAINING A TEXTUAL ERROR MESSAGE
zero. If the error number is unknown, the negative error code PCRE2_ER-
ROR_BADDATA is returned. If the buffer is too small, the message is
truncated (but still with a trailing zero), and the negative error code
- PCRE2_ERROR_NOMEMORY is returned. None of the messages are very long;
- a buffer size of 120 code units is ample.
+ PCRE2_ERROR_NOMEMORY is returned. None of the messages is very long; a
+ buffer size of 120 code units is ample.
+
+
+ITERATING OVER ALL MATCHES
+
+ int pcre2_next_match(pcre2_match_data *match_data,
+ PCRE2_SIZE *pstart_offset, uint32_t *poptions);
+
+ A common task for applications is to implement "global" matching behav-
+ iour, for example, replacing all matches in the subject; splitting the
+ subject on all matches; or simply counting the number of matches. The
+ pcre2_next_match() function helps with this task by providing the ap-
+ propriate parameters for the next match attempt (available since PCRE2
+ 10.46).
+
+ First, a match attempt should be made using one of the matching func-
+ tions (pcre2_match(), pcre2_dfa_match(), or pcre2_jit_match()). Then,
+ pcre2_next_match() can be called, providing the same match_data parame-
+ ter.
+
+ It returns 0 ("false") if there is no need to make a further match at-
+ tempt, or 1 ("true") if another match should be attempted. Returning 1
+ does not imply that there is another match, only that another match
+ should be attempted (which may return PCRE2_ERROR_NOMATCH).
+
+ The *pstart_offset and *poptions are set if the function returns 1.
+ The *pstart_offset should be passed to the next match attempt directly,
+ and the *poptions should be passed to the next match attempt by combin-
+ ing with the application's match options using OR.
+
+ There is some code that demonstrates how to do this in the pcre2demo
+ sample program. The general pattern is:
+
+ uint32_t app_options = ...;
+ uint32_t global_options = 0;
+ PCRE2_SIZE start_offset = 0;
+ while (1)
+ {
+ int rc = pcre2_match(re, subject, subject_len, start_offset,
+ app_options | global_options, match_data,
+ match_context);
+
+ if (rc == PCRE2_ERROR_NOMATCH) break; /* no match, and no more attempts */
+ if (rc < 0) { ... exit }
+
+ ...handle the match
+
+ if (!pcre2_next_match(match_data, &start_offset, &global_options))
+ break; /* no more attempts */
+ }
+
+ The guarantees provided by pcre2_next_match() are that the start_offset
+ will advance, so the loop will definitely terminate. The conditions
+ which ensure this are that either: (a) pcre2_next_match() returns 0
+ (false); or (b) the returned *pstart_offset is strictly greater than
+ the previous start_offset; or (c) if the previous match was a success-
+ ful match of the empty string then the returned *pstart_offset is equal
+ to the previous ovector[1], and *poptions will be set to
+ PCRE2_NOTEMPTY_ATSTART to prevent another empty match from being re-
+ turned.
+
+ A loop implemented as shown above will always terminate, unless there
+ is a bug in PCRE2. As a measure of "defensive programming", applica-
+ tions are encouraged to add an assertion or check to break their loop
+ if it does not make progress (and report the issue as a bug).
+
+ If an application does not use the flag PCRE2_EXTRA_AL-
+ LOW_LOOKAROUND_BSK, then each match is "well-behaved" and satisfies:
+
+ start_offset <= ovector[0] <= ovector[1].
+
+ In this case, the matches found by pcre2_match() with
+ pcre2_next_match() will be sorted, non-overlapping (possibly touching),
+ and with no duplicates.
+
+ Otherwise, if PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK is used, then the guar-
+ antees are considerably weaker. We do not guarantee that the matches
+ will always advance: only that the start_offset will. The matches found
+ by pcre2_match() with pcre2_next_match() will be a finite sequence (as
+ pcre2_next_match() ensures that start_offset advances, so the search
+ will terminate). The matches can however be overlapping, can contain
+ duplicates, and (in truly pathological examples) may not even be sorted
+ by ovector[0]. Additionally, each match itself can end before it starts
+ (ovector[1] < ovector[0]). We recommend that applications do not set
+ PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK.
EXTRACTING CAPTURED SUBSTRINGS BY NUMBER
@@ -3417,39 +3503,39 @@ EXTRACTING CAPTURED SUBSTRINGS BY NUMBER
void pcre2_substring_free(PCRE2_UCHAR *buffer);
- Captured substrings can be accessed directly by using the ovector as
+ Captured substrings can be accessed directly by using the ovector as
described above. For convenience, auxiliary functions are provided for
- extracting captured substrings as new, separate, zero-terminated
+ extracting captured substrings as new, separate, zero-terminated
strings. A substring that contains a binary zero is correctly extracted
- and has a further zero added on the end, but the result is not, of
+ and has a further zero added on the end, but the result is not, of
course, a C string.
The functions in this section identify substrings by number. The number
zero refers to the entire matched substring, with higher numbers refer-
- ring to substrings captured by parenthesized groups. After a partial
- match, only substring zero is available. An attempt to extract any
- other substring gives the error PCRE2_ERROR_PARTIAL. The next section
+ ring to substrings captured by parenthesized groups. After a partial
+ match, only substring zero is available. An attempt to extract any
+ other substring gives the error PCRE2_ERROR_PARTIAL. The next section
describes similar functions for extracting captured substrings by name.
- If a pattern uses the \K escape sequence within a positive assertion,
- the reported start of a successful match can be greater than the end of
- the match. For example, if the pattern (?=ab\K) is matched against
- "ab", the start and end offset values for the match are 2 and 0. In
- this situation, calling these functions with a zero substring number
- extracts a zero-length empty string.
-
- You can find the length in code units of a captured substring without
- extracting it by calling pcre2_substring_length_bynumber(). The first
- argument is a pointer to the match data block, the second is the group
- number, and the third is a pointer to a variable into which the length
- is placed. If you just want to know whether or not the substring has
+ If a pattern uses the \K escape sequence within a positive lookahead
+ assertion, the reported start of a successful match can be greater than
+ the end of the match. For example, if the pattern (?=ab\K) is matched
+ against "ab", the start and end offset values for the match are 2 and
+ 0. In this situation, calling these functions with a zero substring
+ number extracts a zero-length empty string.
+
+ You can find the length in code units of a captured substring without
+ extracting it by calling pcre2_substring_length_bynumber(). The first
+ argument is a pointer to the match data block, the second is the group
+ number, and the third is a pointer to a variable into which the length
+ is placed. If you just want to know whether or not the substring has
been captured, you can pass the third argument as NULL.
- The pcre2_substring_copy_bynumber() function copies a captured sub-
- string into a supplied buffer, whereas pcre2_substring_get_bynumber()
- copies it into new memory, obtained using the same memory allocation
- function that was used for the match data block. The first two argu-
- ments of these functions are a pointer to the match data block and a
+ The pcre2_substring_copy_bynumber() function copies a captured sub-
+ string into a supplied buffer, whereas pcre2_substring_get_bynumber()
+ copies it into new memory, obtained using the same memory allocation
+ function that was used for the match data block. The first two argu-
+ ments of these functions are a pointer to the match data block and a
capture group number.
The final arguments of pcre2_substring_copy_bynumber() are a pointer to
@@ -3458,25 +3544,25 @@ EXTRACTING CAPTURED SUBSTRINGS BY NUMBER
for the extracted substring, excluding the terminating zero.
For pcre2_substring_get_bynumber() the third and fourth arguments point
- to variables that are updated with a pointer to the new memory and the
- number of code units that comprise the substring, again excluding the
- terminating zero. When the substring is no longer needed, the memory
+ to variables that are updated with a pointer to the new memory and the
+ number of code units that comprise the substring, again excluding the
+ terminating zero. When the substring is no longer needed, the memory
should be freed by calling pcre2_substring_free().
- The return value from all these functions is zero for success, or a
- negative error code. If the pattern match failed, the match failure
- code is returned. If a substring number greater than zero is used af-
- ter a partial match, PCRE2_ERROR_PARTIAL is returned. Other possible
+ The return value from all these functions is zero for success, or a
+ negative error code. If the pattern match failed, the match failure
+ code is returned. If a substring number greater than zero is used af-
+ ter a partial match, PCRE2_ERROR_PARTIAL is returned. Other possible
error codes are:
PCRE2_ERROR_NOMEMORY
- The buffer was too small for pcre2_substring_copy_bynumber(), or the
+ The buffer was too small for pcre2_substring_copy_bynumber(), or the
attempt to get memory failed for pcre2_substring_get_bynumber().
PCRE2_ERROR_NOSUBSTRING
- There is no substring with that number in the pattern, that is, the
+ There is no substring with that number in the pattern, that is, the
number is greater than the number of capturing parentheses.
PCRE2_ERROR_UNAVAILABLE
@@ -3487,8 +3573,8 @@ EXTRACTING CAPTURED SUBSTRINGS BY NUMBER
PCRE2_ERROR_UNSET
- The substring did not participate in the match. For example, if the
- pattern is (abc)|(def) and the subject is "def", and the ovector con-
+ The substring did not participate in the match. For example, if the
+ pattern is (abc)|(def) and the subject is "def", and the ovector con-
tains at least two capturing slots, substring number 1 is unset.
@@ -3499,31 +3585,31 @@ EXTRACTING A LIST OF ALL CAPTURED SUBSTRINGS
void pcre2_substring_list_free(PCRE2_UCHAR **list);
- The pcre2_substring_list_get() function extracts all available sub-
- strings and builds a list of pointers to them. It also (optionally)
- builds a second list that contains their lengths (in code units), ex-
- cluding a terminating zero that is added to each of them. All this is
+ The pcre2_substring_list_get() function extracts all available sub-
+ strings and builds a list of pointers to them. It also (optionally)
+ builds a second list that contains their lengths (in code units), ex-
+ cluding a terminating zero that is added to each of them. All this is
done in a single block of memory that is obtained using the same memory
allocation function that was used to get the match data block.
- This function must be called only after a successful match. If called
+ This function must be called only after a successful match. If called
after a partial match, the error code PCRE2_ERROR_PARTIAL is returned.
- The address of the memory block is returned via listptr, which is also
+ The address of the memory block is returned via listptr, which is also
the start of the list of string pointers. The end of the list is marked
- by a NULL pointer. The address of the list of lengths is returned via
- lengthsptr. If your strings do not contain binary zeros and you do not
+ by a NULL pointer. The address of the list of lengths is returned via
+ lengthsptr. If your strings do not contain binary zeros and you do not
therefore need the lengths, you may supply NULL as the lengthsptr argu-
- ment to disable the creation of a list of lengths. The yield of the
- function is zero if all went well, or PCRE2_ERROR_NOMEMORY if the mem-
- ory block could not be obtained. When the list is no longer needed, it
+ ment to disable the creation of a list of lengths. The yield of the
+ function is zero if all went well, or PCRE2_ERROR_NOMEMORY if the mem-
+ ory block could not be obtained. When the list is no longer needed, it
should be freed by calling pcre2_substring_list_free().
If this function encounters a substring that is unset, which can happen
- when capture group number n+1 matches some part of the subject, but
- group n has not been used at all, it returns an empty string. This can
+ when capture group number n+1 matches some part of the subject, but
+ group n has not been used at all, it returns an empty string. This can
be distinguished from a genuine zero-length substring by inspecting the
- appropriate offset in the ovector, which contain PCRE2_UNSET for unset
+ appropriate offset in the ovector, which contain PCRE2_UNSET for unset
substrings, or by calling pcre2_substring_length_bynumber().
@@ -3543,7 +3629,7 @@ EXTRACTING CAPTURED SUBSTRINGS BY NAME
void pcre2_substring_free(PCRE2_UCHAR *buffer);
- To extract a substring by name, you first have to find associated num-
+ To extract a substring by name, you first have to find associated num-
ber. For example, for this pattern:
(a+)b(?\d+)...
@@ -3551,32 +3637,32 @@ EXTRACTING CAPTURED SUBSTRINGS BY NAME
the number of the capture group called "xxx" is 2. If the name is known
to be unique (PCRE2_DUPNAMES was not set), you can find the number from
the name by calling pcre2_substring_number_from_name(). The first argu-
- ment is the compiled pattern, and the second is the name. The yield of
- the function is the group number, PCRE2_ERROR_NOSUBSTRING if there is
- no group with that name, or PCRE2_ERROR_NOUNIQUESUBSTRING if there is
- more than one group with that name. Given the number, you can extract
- the substring directly from the ovector, or use one of the "bynumber"
+ ment is the compiled pattern, and the second is the name. The yield of
+ the function is the group number, PCRE2_ERROR_NOSUBSTRING if there is
+ no group with that name, or PCRE2_ERROR_NOUNIQUESUBSTRING if there is
+ more than one group with that name. Given the number, you can extract
+ the substring directly from the ovector, or use one of the "bynumber"
functions described above.
- For convenience, there are also "byname" functions that correspond to
+ For convenience, there are also "byname" functions that correspond to
the "bynumber" functions, the only difference being that the second ar-
- gument is a name instead of a number. If PCRE2_DUPNAMES is set and
+ gument is a name instead of a number. If PCRE2_DUPNAMES is set and
there are duplicate names, these functions scan all the groups with the
- given name, and return the captured substring from the first named
+ given name, and return the captured substring from the first named
group that is set.
- If there are no groups with the given name, PCRE2_ERROR_NOSUBSTRING is
- returned. If all groups with the name have numbers that are greater
+ If there are no groups with the given name, PCRE2_ERROR_NOSUBSTRING is
+ returned. If all groups with the name have numbers that are greater
than the number of slots in the ovector, PCRE2_ERROR_UNAVAILABLE is re-
- turned. If there is at least one group with a slot in the ovector, but
+ turned. If there is at least one group with a slot in the ovector, but
no group is found to be set, PCRE2_ERROR_UNSET is returned.
Warning: If the pattern uses the (?| feature to set up multiple capture
- groups with the same number, as described in the section on duplicate
+ groups with the same number, as described in the section on duplicate
group numbers in the pcre2pattern page, you cannot use names to distin-
- guish the different capture groups, because names are not included in
- the compiled code. The matching process uses only numbers. For this
- reason, the use of different names for groups with the same number
+ guish the different capture groups, because names are not included in
+ the compiled code. The matching process uses only numbers. For this
+ reason, the use of different names for groups with the same number
causes an error at compile time.
@@ -3589,30 +3675,32 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
PCRE2_SIZE rlength, PCRE2_UCHAR *outputbuffer,
PCRE2_SIZE *outlengthptr);
- This function optionally calls pcre2_match() and then makes a copy of
- the subject string in outputbuffer, replacing parts that were matched
+ This function optionally calls pcre2_match() and then makes a copy of
+ the subject string in outputbuffer, replacing parts that were matched
with the replacement string, whose length is supplied in rlength, which
- can be given as PCRE2_ZERO_TERMINATED for a zero-terminated string. As
- a special case, if replacement is NULL and rlength is zero, the re-
- placement is assumed to be an empty string. If rlength is non-zero, an
+ can be given as PCRE2_ZERO_TERMINATED for a zero-terminated string. As
+ a special case, if replacement is NULL and rlength is zero, the re-
+ placement is assumed to be an empty string. If rlength is non-zero, an
error occurs if replacement is NULL.
There is an option (see PCRE2_SUBSTITUTE_REPLACEMENT_ONLY below) to re-
- turn just the replacement string(s). The default action is to perform
- just one replacement if the pattern matches, but there is an option
- that requests multiple replacements (see PCRE2_SUBSTITUTE_GLOBAL be-
+ turn just the replacement string(s). The default action is to perform
+ just one replacement if the pattern matches, but there is an option
+ that requests multiple replacements (see PCRE2_SUBSTITUTE_GLOBAL be-
low).
- If successful, pcre2_substitute() returns the number of substitutions
- that were carried out. This may be zero if no match was found, and is
- never greater than one unless PCRE2_SUBSTITUTE_GLOBAL is set. A nega-
+ If successful, pcre2_substitute() returns the number of substitutions
+ that were carried out. This may be zero if no match was found, and is
+ never greater than one unless PCRE2_SUBSTITUTE_GLOBAL is set. A nega-
tive value is returned if an error is detected.
- Matches in which a \K item in a lookahead in the pattern causes the
- match to end before it starts are not supported, and give rise to an
+ Matches in which a \K item in a lookahead in the pattern causes the
+ match to end before it starts are not supported, and give rise to an
error return. For global replacements, matches in which \K in a lookbe-
- hind causes the match to start earlier than the point that was reached
- in the previous iteration are also not supported.
+ hind causes the match to start earlier than the point that was reached
+ in the previous iteration are also not supported. (These cases are only
+ possible if the pattern was compiled with the backwards-compatibility
+ option PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK.)
The first seven arguments of pcre2_substitute() are the same as for
pcre2_match(), except that the partial matching options are not permit-
@@ -3623,79 +3711,102 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
If match_data is not NULL and PCRE2_SUBSTITUTE_MATCHED is not set, the
provided block is used for all calls to pcre2_match(), and its contents
- afterwards are the result of the final call. For global changes, this
+ afterwards are the result of the final call made internally by
+ pcre2_substitute() to the matching function. For global changes, this
will always be a no-match error. The contents of the ovector within the
match data block may or may not have been changed.
- As well as the usual options for pcre2_match(), a number of additional
- options can be set in the options argument of pcre2_substitute(). One
- such option is PCRE2_SUBSTITUTE_MATCHED. When this is set, an external
- match_data block must be provided, and it must have already been used
- for an external call to pcre2_match() with the same pattern and subject
- arguments. The data in the match_data block (return code, offset vec-
- tor) is then used for the first substitution instead of calling
- pcre2_match() from within pcre2_substitute(). This allows an applica-
- tion to check for a match before choosing to substitute, without having
- to repeat the match.
+ As well as the usual options for pcre2_match(), a number of additional
+ options can be set in the options argument of pcre2_substitute(). One
+ such option is PCRE2_SUBSTITUTE_MATCHED. When this is set, an external
+ match_data block must be provided, and it must have already been used
+ for an external call to pcre2_match() (or pcre2_jit_match()) with the
+ same pattern, subject pointer, effective subject length, start offset,
+ and match option arguments (substitute-specific options can be added to
+ the options argument). If any of these parameters is changed,
+ pcre2_substitute() returns an error. The data in the match_data block
+ (return code, offset vector) is used for the first substitution instead
+ of calling pcre2_match() from within pcre2_substitute(). This allows an
+ application to check for a match before choosing to substitute, without
+ having to repeat the match.
+
+ If the contents of the subject buffer are mutated in between
+ pcre2_match() and a call to pcre2_substitute() with PCRE2_SUBSTI-
+ TUTE_MATCHED, the behaviour is unsafe; in particular, in this case,
+ PCRE2 is unable to ensure that the offsets in the ovector point to the
+ start of characters (with UTF-encoded input).
The contents of the externally supplied match data block are not
- changed when PCRE2_SUBSTITUTE_MATCHED is set. If PCRE2_SUBSTI-
- TUTE_GLOBAL is also set, pcre2_match() is called after the first sub-
- stitution to check for further matches, but this is done using an in-
- ternally obtained match data block, thus always leaving the external
- block unchanged.
-
- The code argument is not used for matching before the first substitu-
- tion when PCRE2_SUBSTITUTE_MATCHED is set, but it must be provided,
- even when PCRE2_SUBSTITUTE_GLOBAL is not set, because it contains in-
+ changed when PCRE2_SUBSTITUTE_MATCHED is set, and so the match block is
+ permitted for use in another call using PCRE2_SUBSTITUTE_MATCHED. If
+ PCRE2_SUBSTITUTE_GLOBAL is also set, pcre2_match() is called after the
+ first substitution to check for furthe matches, but this is done using
+ an internally obtained match data block, thus always leaving the exter-
+ nal block unchanged.
+
+ The code argument is not used for matching before the first substitu-
+ tion when PCRE2_SUBSTITUTE_MATCHED is set, but it must be provided,
+ even when PCRE2_SUBSTITUTE_GLOBAL is not set, because it contains in-
formation such as the UTF setting and the number of capturing parenthe-
ses in the pattern.
- The default action of pcre2_substitute() is to return a copy of the
+ When using PCRE2_SUBSTITUTE_MATCHED, you should not modify the subject
+ string in between the prior call to pcre2_match() and pcre2_substi-
+ tute(), as the substitution assumes that the passed-in ovector is com-
+ patible with the subject string. Although PCRE2 does verify that the
+ subject is a pointer to the same buffer, it cannot in general verify
+ whether the contents of the buffer have changed. For example, if the
+ subject buffer is mutated from one valid UTF-8 string to another valid
+ string, of the same length in code units, the ovector offsets are no
+ longer guaranteed to point to the start of a character. Beware that
+ with PCRE2_SUBSTITUTE_MATCHED in UTF mode, the subject string is not
+ re-scanned for UTF validity when pcre2_substitute() first uses it.
+
+ The default action of pcre2_substitute() is to return a copy of the
subject string with matched substrings replaced. However, if PCRE2_SUB-
- STITUTE_REPLACEMENT_ONLY is set, only the replacement substrings are
+ STITUTE_REPLACEMENT_ONLY is set, only the replacement substrings are
returned. In the global case, multiple replacements are concatenated in
- the output buffer. Substitution callouts (see below) can be used to
+ the output buffer. Substitution callouts (see below) can be used to
separate them if necessary.
- The outlengthptr argument of pcre2_substitute() must point to a vari-
- able that contains the length, in code units, of the output buffer. If
- the function is successful, the value is updated to contain the length
- in code units of the new string, excluding the trailing zero that is
+ The outlengthptr argument of pcre2_substitute() must point to a vari-
+ able that contains the length, in code units, of the output buffer. If
+ the function is successful, the value is updated to contain the length
+ in code units of the new string, excluding the trailing zero that is
automatically added.
- If the function is not successful, the value set via outlengthptr de-
- pends on the type of error. For syntax errors in the replacement
+ If the function is not successful, the value set via outlengthptr de-
+ pends on the type of error. For syntax errors in the replacement
string, the value is the offset in the replacement string where the er-
- ror was detected. For other errors, the value is PCRE2_UNSET by de-
+ ror was detected. For other errors, the value is PCRE2_UNSET by de-
fault. This includes the case of the output buffer being too small, un-
less PCRE2_SUBSTITUTE_OVERFLOW_LENGTH is set.
- PCRE2_SUBSTITUTE_OVERFLOW_LENGTH changes what happens when the output
+ PCRE2_SUBSTITUTE_OVERFLOW_LENGTH changes what happens when the output
buffer is too small. The default action is to return PCRE2_ERROR_NOMEM-
- ORY immediately. If this option is set, however, pcre2_substitute()
+ ORY immediately. If this option is set, however, pcre2_substitute()
continues to go through the motions of matching and substituting (with-
- out, of course, writing anything) in order to compute the size of
- buffer that is needed, which will include the extra space for the ter-
- minating NUL. This value is passed back via the outlengthptr variable,
+ out, of course, writing anything) in order to compute the size of
+ buffer that is needed, which will include the extra space for the ter-
+ minating NUL. This value is passed back via the outlengthptr variable,
with the result of the function still being PCRE2_ERROR_NOMEMORY.
- Passing a buffer size of zero is a permitted way of finding out how
- much memory is needed for given substitution. However, this does mean
+ Passing a buffer size of zero is a permitted way of finding out how
+ much memory is needed for given substitution. However, this does mean
that the entire operation is carried out twice. Depending on the appli-
- cation, it may be more efficient to allocate a large buffer and free
- the excess afterwards, instead of using PCRE2_SUBSTITUTE_OVER-
+ cation, it may be more efficient to allocate a large buffer and free
+ the excess afterwards, instead of using PCRE2_SUBSTITUTE_OVER-
FLOW_LENGTH.
- The replacement string, which is interpreted as a UTF string in UTF
- mode, is checked for UTF validity unless PCRE2_NO_UTF_CHECK is set. An
+ The replacement string, which is interpreted as a UTF string in UTF
+ mode, is checked for UTF validity unless PCRE2_NO_UTF_CHECK is set. An
invalid UTF replacement string causes an immediate return with the rel-
evant UTF error code.
- If PCRE2_SUBSTITUTE_LITERAL is set, the replacement string is not in-
+ If PCRE2_SUBSTITUTE_LITERAL is set, the replacement string is not in-
terpreted in any way. By default, however, a dollar character is an es-
- cape character that can specify the insertion of characters from cap-
- ture groups and names from (*MARK) or other control verbs in the pat-
+ cape character that can specify the insertion of characters from cap-
+ ture groups and names from (*MARK) or other control verbs in the pat-
tern. Dollar is the only escape character (backslash is treated as lit-
eral). The following forms are recognized:
@@ -3705,6 +3816,8 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
$` insert the substring that precedes the match
$' insert the substring that follows the match
$_ insert the entire input string
+ $+ insert the highest-numbered capture group which
+ matched
$*MARK or ${*MARK} insert a control verb name
Either a group number or a group name can be given for n, for example
@@ -3738,6 +3851,18 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
subject string. If an offset limit is set in the match context, search-
ing stops when that limit is reached.
+ Because global substitutions apply the pattern repeatedly to the sub-
+ ject string, and always iterate over non-overlapping matches, the sub-
+ stitutions done by pcre2_substitute() do not match and substitute text
+ inside the replacement strings themselves (no recursive/iterative sub-
+ stitution). However, applications can easily implement other alterna-
+ tive replacement strategies, such as iteratively replacing, then match-
+ ing and replacing on the result. The replacement loop inside pcre2_sub-
+ stitute() is simple and can be emulated in client code by allocating a
+ buffer, searching for matches in a loop, and calling pcre2_substitute()
+ with PCRE2_SUBSTITUTE_REPLACEMENT_ONLY an PCRE2_SUBSTITUTE_MATCHED, and
+ without PCRE2_SUBSTITUTE_GLOBAL.
+
You can restrict the effect of a global substitution to a portion of
the subject string by setting either or both of startoffset and an off-
set limit. Here is a pcre2test example:
@@ -3782,59 +3907,59 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
given in the pcre2pattern page.
The Python form \g, where the angle brackets are part of the syntax
- and n is either a group name or number, is recognized as an altertive
+ and n is either a group name or number, is recognized as an alternative
way of inserting the contents of a group, for example \g<3>.
- There are also four escape sequences for forcing the case of inserted
- letters. Case forcing applies to all inserted characters, including
- those from capture groups and letters within \Q...\E quoted sequences.
- The insertion mechanism has three states: no case forcing, force upper
- case, and force lower case. The escape sequences change the current
- state: \U and \L change to upper or lower case forcing, respectively,
- and \E (when not terminating a \Q quoted sequence) reverts to no case
- forcing. The sequences \u and \l force the next character (if it is a
- letter) to upper or lower case, respectively, and then the state auto-
+ There are also four escape sequences for forcing the case of inserted
+ letters. Case forcing applies to all inserted characters, including
+ those from capture groups and letters within \Q...\E quoted sequences.
+ The insertion mechanism has three states: no case forcing, force upper
+ case, and force lower case. The escape sequences change the current
+ state: \U and \L change to upper or lower case forcing, respectively,
+ and \E (when not terminating a \Q quoted sequence) reverts to no case
+ forcing. The sequences \u and \l force the next character (if it is a
+ letter) to upper or lower case, respectively, and then the state auto-
matically reverts to no case forcing.
- However, if \u is immediately followed by \L or \l is immediately fol-
- lowed by \U, the next character's case is forced by the first escape
+ However, if \u is immediately followed by \L or \l is immediately fol-
+ lowed by \U, the next character's case is forced by the first escape
sequence, and subsequent characters by the second. This provides a "ti-
- tle casing" facility that can be applied to group captures. For exam-
- ple, if group 1 has captured "heLLo", the replacement string "\u\L$1"
+ tle casing" facility that can be applied to group captures. For exam-
+ ple, if group 1 has captured "heLLo", the replacement string "\u\L$1"
becomes "Hello".
If either PCRE2_UTF or PCRE2_UCP was set when the pattern was compiled,
- Unicode properties are used for case forcing characters whose code
- points are greater than 127. However, only simple case folding, as de-
- termined by the Unicode file CaseFolding.txt is supported. PCRE2 does
- not support language-specific special casing rules such as using dif-
- ferent lower case Greek sigmas in the middle and ends of words (as de-
+ Unicode properties are used for case forcing characters whose code
+ points are greater than 127. However, only simple case folding, as de-
+ termined by the Unicode file CaseFolding.txt is supported. PCRE2 does
+ not support language-specific special casing rules such as using dif-
+ ferent lower case Greek sigmas in the middle and ends of words (as de-
fined in the Unicode file SpecialCasing.txt).
Note that case forcing sequences such as \U...\E do not nest. For exam-
- ple, the result of processing "\Uaa\LBB\Ecc\E" is "AAbbcc"; the final
- \E has no effect. Note also that the PCRE2_ALT_BSUX and PCRE2_EX-
+ ple, the result of processing "\Uaa\LBB\Ecc\E" is "AAbbcc"; the final
+ \E has no effect. Note also that the PCRE2_ALT_BSUX and PCRE2_EX-
TRA_ALT_BSUX options do not apply to replacement strings.
- The final effect of setting PCRE2_SUBSTITUTE_EXTENDED is to add more
- flexibility to capture group substitution. The syntax is similar to
+ The final effect of setting PCRE2_SUBSTITUTE_EXTENDED is to add more
+ flexibility to capture group substitution. The syntax is similar to
that used by Bash:
${n:-string}
${n:+string1:string2}
- As in the simple case, n may be a group number or a name. The first
- form specifies a default value. If group n is set, its value is in-
- serted; if not, the string is expanded and the result inserted. The
+ As in the simple case, n may be a group number or a name. The first
+ form specifies a default value. If group n is set, its value is in-
+ serted; if not, the string is expanded and the result inserted. The
second form specifies strings that are expanded and inserted when group
- n is set or unset, respectively. The first form is just a convenient
+ n is set or unset, respectively. The first form is just a convenient
shorthand for
${n:+${n}:string}
- Backslash can be used to escape colons and closing curly brackets in
- the replacement strings. A change of the case forcing state within a
- replacement string remains in force afterwards, as shown in this
+ Backslash can be used to escape colons and closing curly brackets in
+ the replacement strings. A change of the case forcing state within a
+ replacement string remains in force afterwards, as shown in this
pcre2test example:
/(some)?(body)/substitute_extended,replace=${1:+\U:\L}HeLLo
@@ -3843,8 +3968,8 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
somebody
1: HELLO
- The PCRE2_SUBSTITUTE_UNSET_EMPTY option does not affect these extended
- substitutions. However, PCRE2_SUBSTITUTE_UNKNOWN_UNSET does cause un-
+ The PCRE2_SUBSTITUTE_UNSET_EMPTY option does not affect these extended
+ substitutions. However, PCRE2_SUBSTITUTE_UNKNOWN_UNSET does cause un-
known groups in the extended syntax forms to be treated as unset.
If PCRE2_SUBSTITUTE_LITERAL is set, PCRE2_SUBSTITUTE_UNKNOWN_UNSET,
@@ -3853,39 +3978,39 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
Substitution errors
- In the event of an error, pcre2_substitute() returns a negative error
- code. Except for PCRE2_ERROR_NOMATCH (which is never returned), errors
+ In the event of an error, pcre2_substitute() returns a negative error
+ code. Except for PCRE2_ERROR_NOMATCH (which is never returned), errors
from pcre2_match() are passed straight back.
PCRE2_ERROR_NOSUBSTRING is returned for a non-existent substring inser-
tion, unless PCRE2_SUBSTITUTE_UNKNOWN_UNSET is set.
PCRE2_ERROR_UNSET is returned for an unset substring insertion (includ-
- ing an unknown substring when PCRE2_SUBSTITUTE_UNKNOWN_UNSET is set)
- when the simple (non-extended) syntax is used and PCRE2_SUBSTITUTE_UN-
+ ing an unknown substring when PCRE2_SUBSTITUTE_UNKNOWN_UNSET is set)
+ when the simple (non-extended) syntax is used and PCRE2_SUBSTITUTE_UN-
SET_EMPTY is not set.
- PCRE2_ERROR_NOMEMORY is returned if the output buffer is not big
+ PCRE2_ERROR_NOMEMORY is returned if the output buffer is not big
enough. If the PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option is set, the size
- of buffer that is needed is returned via outlengthptr. Note that this
+ of buffer that is needed is returned via outlengthptr. Note that this
does not happen by default.
PCRE2_ERROR_NULL is returned if PCRE2_SUBSTITUTE_MATCHED is set but the
- match_data argument is NULL or if the subject or replacement arguments
- are NULL. For backward compatibility reasons an exception is made for
+ match_data argument is NULL or if the subject or replacement arguments
+ are NULL. For backward compatibility reasons an exception is made for
the replacement argument if the rlength argument is also 0.
- PCRE2_ERROR_BADREPLACEMENT is used for miscellaneous syntax errors in
- the replacement string, with more particular errors being PCRE2_ER-
+ PCRE2_ERROR_BADREPLACEMENT is used for miscellaneous syntax errors in
+ the replacement string, with more particular errors being PCRE2_ER-
ROR_BADREPESCAPE (invalid escape sequence), PCRE2_ERROR_REPMISSINGBRACE
- (closing curly bracket not found), PCRE2_ERROR_BADSUBSTITUTION (syntax
- error in extended group substitution), and PCRE2_ERROR_BADSUBSPATTERN
+ (closing curly bracket not found), PCRE2_ERROR_BADSUBSTITUTION (syntax
+ error in extended group substitution), and PCRE2_ERROR_BADSUBSPATTERN
(the pattern match ended before it started or the match started earlier
- than the current position in the subject, which can happen if \K is
- used in an assertion).
+ than the current position in the subject, which can happen if \K is
+ used in a lookaround assertion).
As for all PCRE2 errors, a text message that describes the error can be
- obtained by calling the pcre2_get_error_message() function (see "Ob-
+ obtained by calling the pcre2_get_error_message() function (see "Ob-
taining a textual error message" above).
Substitution callouts
@@ -3894,23 +4019,23 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
int (*callout_function)(pcre2_substitute_callout_block *, void *),
void *callout_data);
- The pcre2_set_substitution_callout() function can be used to specify a
- callout function for pcre2_substitute(). This information is passed in
+ The pcre2_set_substitute_callout() function can be used to specify a
+ callout function for pcre2_substitute(). This information is passed in
a match context. The callout function is called after each substitution
has been processed, but it can cause the replacement not to happen.
- The callout function is not called for simulated substitutions that
- happen as a result of the PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option. In
- this mode, when substitution processing exceeds the buffer space pro-
- vided by the caller, processing continues by counting code units. The
- simulation is unable to populate the callout block, and so the simula-
+ The callout function is not called for simulated substitutions that
+ happen as a result of the PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option. In
+ this mode, when substitution processing exceeds the buffer space pro-
+ vided by the caller, processing continues by counting code units. The
+ simulation is unable to populate the callout block, and so the simula-
tion is pessimistic about the required buffer size. Whichever is larger
- of accepted or rejected substitution is reported as the required size.
+ of accepted or rejected substitution is reported as the required size.
Therefore, the returned buffer length may be an overestimate (without a
substitution callout, it is normally an exact measurement).
The first argument of the callout function is a pointer to a substitute
- callout block structure, which contains the following fields, not nec-
+ callout block structure, which contains the following fields, not nec-
essarily in this order:
uint32_t version;
@@ -3921,34 +4046,34 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
uint32_t oveccount;
PCRE2_SIZE output_offsets[2];
- The version field contains the version number of the block format. The
- current version is 0. The version number will increase in future if
- more fields are added, but the intention is never to remove any of the
+ The version field contains the version number of the block format. The
+ current version is 0. The version number will increase in future if
+ more fields are added, but the intention is never to remove any of the
existing fields.
The subscount field is the number of the current match. It is 1 for the
first callout, 2 for the second, and so on. The input and output point-
ers are copies of the values passed to pcre2_substitute().
- The ovector field points to the ovector, which contains the result of
+ The ovector field points to the ovector, which contains the result of
the most recent match. The oveccount field contains the number of pairs
that are set in the ovector, and is always greater than zero.
- The output_offsets vector contains the offsets of the replacement in
- the output string. This has already been processed for dollar and (if
+ The output_offsets vector contains the offsets of the replacement in
+ the output string. This has already been processed for dollar and (if
requested) backslash substitutions as described above.
- The second argument of the callout function is the value passed as
- callout_data when the function was registered. The value returned by
+ The second argument of the callout function is the value passed as
+ callout_data when the function was registered. The value returned by
the callout function is interpreted as follows:
- If the value is zero, the replacement is accepted, and, if PCRE2_SUB-
- STITUTE_GLOBAL is set, processing continues with a search for the next
- match. If the value is not zero, the current replacement is not ac-
- cepted. If the value is greater than zero, processing continues when
- PCRE2_SUBSTITUTE_GLOBAL is set. Otherwise (the value is less than zero
+ If the value is zero, the replacement is accepted, and, if PCRE2_SUB-
+ STITUTE_GLOBAL is set, processing continues with a search for the next
+ match. If the value is not zero, the current replacement is not ac-
+ cepted. If the value is greater than zero, processing continues when
+ PCRE2_SUBSTITUTE_GLOBAL is set. Otherwise (the value is less than zero
or PCRE2_SUBSTITUTE_GLOBAL is not set), the rest of the input is copied
- to the output and the call to pcre2_substitute() exits, returning the
+ to the output and the call to pcre2_substitute() exits, returning the
number of matches so far.
Substitution case callouts
@@ -3959,11 +4084,11 @@ CREATING A NEW STRING WITH SUBSTITUTIONS
int, void *),
void *callout_data);
- The pcre2_set_substitution_case_callout() function can be used to spec-
- ify a callout function for pcre2_substitute() to use when performing
- case transformations. This does not affect any case insensitivity be-
- haviour when performing a match, but only the user-visible transforma-
- tions performed when processing a substitution such as:
+ The pcre2_set_substitute_case_callout() function can be used to specify
+ a callout function for pcre2_substitute() to use when performing case
+ transformations. This does not affect any case insensitivity behaviour
+ when performing a match, but only the user-visible transformations per-
+ formed when processing a substitution such as:
pcre2_substitute(..., "\\U$1", ...)
@@ -4293,11 +4418,11 @@ AUTHOR
REVISION
- Last updated: 26 December 2024
+ Last updated: 19 October 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 26 December 2024 PCRE2API(3)
+PCRE2 10.47 19 October 2025 PCRE2API(3)
------------------------------------------------------------------------------
@@ -4311,43 +4436,43 @@ NAME
BUILDING PCRE2
PCRE2 is distributed with a configure script that can be used to build
- the library in Unix-like environments using the applications known as
- Autotools. Also in the distribution are files to support building using
- CMake instead of configure. The text file README contains general in-
- formation about building with Autotools (some of which is repeated be-
- low), and also has some comments about building on various operating
- systems. The files in the vms directory support building under OpenVMS.
- There is a lot more information about building PCRE2 without using Au-
- totools (including information about using CMake and building "by
- hand") in the text file called NON-AUTOTOOLS-BUILD. You should consult
- this file as well as the README file if you are building in a non-Unix-
- like environment.
+ the library in Unix-like environments using the Autotools applications.
+ Also in the distribution are files to support building using CMake in-
+ stead of configure. The text file README contains general information
+ about building with Autotools (some of which is repeated below), and
+ also has some comments about building on various operating systems. The
+ files in the vms directory support building under OpenVMS. There is a
+ lot more information about building PCRE2 without using Autotools (in-
+ cluding information about using CMake and building "by hand") in the
+ text file called NON-AUTOTOOLS-BUILD. You should consult this file as
+ well as the README file if you are building in a non-Unix-like environ-
+ ment.
PCRE2 BUILD-TIME OPTIONS
The rest of this document describes the optional features of PCRE2 that
- can be selected when the library is compiled. It assumes use of the
- configure script, where the optional features are selected or dese-
- lected by providing options to configure before running the make com-
- mand. However, the same options can be selected in both Unix-like and
- non-Unix-like environments if you are using CMake instead of configure
+ can be selected when the library is compiled. It assumes use of the
+ configure script, where the optional features are selected or dese-
+ lected by providing options to configure before running the make com-
+ mand. However, the same options can be selected in both Unix-like and
+ non-Unix-like environments if you are using CMake instead of configure
to build PCRE2.
- If you are not using Autotools or CMake, option selection can be done
- by editing the config.h file, or by passing parameter settings to the
+ If you are not using Autotools or CMake, option selection can be done
+ by editing the config.h file, or by passing parameter settings to the
compiler, as described in NON-AUTOTOOLS-BUILD.
The complete list of options for configure (which includes the standard
- ones such as the selection of the installation directory) can be ob-
+ ones such as the selection of the installation directory) can be ob-
tained by running
./configure --help
- The following sections include descriptions of "on/off" options whose
+ The following sections include descriptions of "on/off" options whose
names begin with --enable or --disable. Because of the way that config-
- ure works, --enable and --disable always come in pairs, so the comple-
- mentary option always exists as well, but as it specifies the default,
+ ure works, --enable and --disable always come in pairs, so the comple-
+ mentary option always exists as well, but as it specifies the default,
it is not described. Options that specify values have names that start
with --with. At the end of a configure run, a summary of the configura-
tion is output.
@@ -4355,13 +4480,13 @@ PCRE2 BUILD-TIME OPTIONS
BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES
- By default, a library called libpcre2-8 is built, containing functions
- that take string arguments contained in arrays of bytes, interpreted
- either as single-byte characters, or UTF-8 strings. You can also build
- two other libraries, called libpcre2-16 and libpcre2-32, which process
- strings that are contained in arrays of 16-bit and 32-bit code units,
+ By default, a library called libpcre2-8 is built, containing functions
+ that take string arguments contained in arrays of bytes, interpreted
+ either as single-byte characters, or UTF-8 strings. You can also build
+ two other libraries, called libpcre2-16 and libpcre2-32, which process
+ strings that are contained in arrays of 16-bit and 32-bit code units,
respectively. These can be interpreted either as single-unit characters
- or UTF-16/UTF-32 strings. To build these additional libraries, add one
+ or UTF-16/UTF-32 strings. To build these additional libraries, add one
or both of the following to the configure command:
--enable-pcre2-16
@@ -4371,71 +4496,71 @@ BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES
--disable-pcre2-8
- as well. At least one of the three libraries must be built. Note that
- the POSIX wrapper is for the 8-bit library only, and that pcre2grep is
- an 8-bit program. Neither of these are built if you select only the
+ as well. At least one of the three libraries must be built. Note that
+ the POSIX wrapper is for the 8-bit library only, and that pcre2grep is
+ an 8-bit program. Neither of these are built if you select only the
16-bit or 32-bit libraries.
BUILDING SHARED AND STATIC LIBRARIES
- The Autotools PCRE2 building process uses libtool to build both shared
- and static libraries by default. You can suppress an unwanted library
+ The Autotools PCRE2 building process uses libtool to build both shared
+ and static libraries by default. You can suppress an unwanted library
by adding one of
--disable-shared
--disable-static
- to the configure command. Setting --disable-shared ensures that PCRE2
- libraries are built as static libraries. The binaries that are then
- created as part of the build process (for example, pcre2test and
- pcre2grep) are linked statically with one or more PCRE2 libraries, but
- may also be dynamically linked with other libraries such as libc. If
- you want these binaries to be fully statically linked, you can set LD-
+ to the configure command. Setting --disable-shared ensures that PCRE2
+ libraries are built as static libraries. The binaries that are then
+ created as part of the build process (for example, pcre2test and
+ pcre2grep) are linked statically with one or more PCRE2 libraries, but
+ may also be dynamically linked with other libraries such as libc. If
+ you want these binaries to be fully statically linked, you can set LD-
FLAGS like this:
LDFLAGS=--static ./configure --disable-shared
- Note the two hyphens in --static. Of course, this works only if static
+ Note the two hyphens in --static. Of course, this works only if static
versions of all the relevant libraries are available for linking.
UNICODE AND UTF SUPPORT
- By default, PCRE2 is built with support for Unicode and UTF character
+ By default, PCRE2 is built with support for Unicode and UTF character
strings. To build it without Unicode support, add
--disable-unicode
- to the configure command. This setting applies to all three libraries.
- It is not possible to build one library with Unicode support and an-
+ to the configure command. This setting applies to all three libraries.
+ It is not possible to build one library with Unicode support and an-
other without in the same configuration.
- Of itself, Unicode support does not make PCRE2 treat strings as UTF-8,
+ Of itself, Unicode support does not make PCRE2 treat strings as UTF-8,
UTF-16 or UTF-32. To do that, applications that use the library can set
- the PCRE2_UTF option when they call pcre2_compile() to compile a pat-
- tern. Alternatively, patterns may be started with (*UTF) unless the
+ the PCRE2_UTF option when they call pcre2_compile() to compile a pat-
+ tern. Alternatively, patterns may be started with (*UTF) unless the
application has locked this out by setting PCRE2_NEVER_UTF.
UTF support allows the libraries to process character code points up to
- 0x10ffff in the strings that they handle. Unicode support also gives
- access to the Unicode properties of characters, using pattern escapes
+ 0x10ffff in the strings that they handle. Unicode support also gives
+ access to the Unicode properties of characters, using pattern escapes
such as \P, \p, and \X. Only the general category properties such as Lu
- and Nd, script names, and some bi-directional properties are supported.
- Details are given in the pcre2pattern documentation.
+ and Nd, script names, and some bi-directional and binary properties are
+ supported. Details are given in the pcre2pattern documentation.
Pattern escapes such as \d and \w do not by default make use of Unicode
- properties. The application can request that they do by setting the
- PCRE2_UCP option. Unless the application has set PCRE2_NEVER_UCP, a
+ properties. The application can request that they do by setting the
+ PCRE2_UCP option. Unless the application has set PCRE2_NEVER_UCP, a
pattern may also request this by starting with (*UCP).
DISABLING THE USE OF \C
The \C escape sequence, which matches a single code unit, even in a UTF
- mode, can cause unpredictable behaviour because it may leave the cur-
- rent matching point in the middle of a multi-code-unit character. The
- application can lock it out by setting the PCRE2_NEVER_BACKSLASH_C op-
+ mode, can cause unpredictable behaviour because it may leave the cur-
+ rent matching point in the middle of a multi-code-unit character. The
+ application can lock it out by setting the PCRE2_NEVER_BACKSLASH_C op-
tion when calling pcre2_compile(). There is also a build-time option
--enable-never-backslash-C
@@ -4445,27 +4570,27 @@ DISABLING THE USE OF \C
JUST-IN-TIME COMPILER SUPPORT
- Just-in-time (JIT) compiler support is included in the build by speci-
+ Just-in-time (JIT) compiler support is included in the build by speci-
fying
--enable-jit
- This support is available only for certain hardware architectures. If
- this option is set for an unsupported architecture, a building error
+ This support is available only for certain hardware architectures. If
+ this option is set for an unsupported architecture, a building error
occurs. If in doubt, use
--enable-jit=auto
- which enables JIT only if the current hardware is supported. You can
- check if JIT is enabled in the configuration summary that is output at
- the end of a configure run. If you are enabling JIT under SELinux you
+ which enables JIT only if the current hardware is supported. You can
+ check if JIT is enabled in the configuration summary that is output at
+ the end of a configure run. If you are enabling JIT under SELinux you
may also want to add
--enable-jit-sealloc
which enables the use of an execmem allocator in JIT that is compatible
- with SELinux. This has no effect if JIT is not enabled. See the
- pcre2jit documentation for a discussion of JIT usage. When JIT support
+ with SELinux. This has no effect if JIT is not enabled. See the
+ pcre2jit documentation for a discussion of JIT usage. When JIT support
is enabled, pcre2grep automatically makes use of it, unless you add
--disable-pcre2grep-jit
@@ -4475,14 +4600,14 @@ JUST-IN-TIME COMPILER SUPPORT
NEWLINE RECOGNITION
- By default, PCRE2 interprets the linefeed (LF) character as indicating
- the end of a line. This is the normal newline character on Unix-like
- systems. You can compile PCRE2 to use carriage return (CR) instead, by
+ By default, PCRE2 interprets the linefeed (LF) character as indicating
+ the end of a line. This is the normal newline character on Unix-like
+ systems. You can compile PCRE2 to use carriage return (CR) instead, by
adding
--enable-newline-is-cr
- to the configure command. There is also an --enable-newline-is-lf op-
+ to the configure command. There is also an --enable-newline-is-lf op-
tion, which explicitly specifies linefeed as the newline character.
Alternatively, you can specify that line endings are to be indicated by
@@ -4495,126 +4620,126 @@ NEWLINE RECOGNITION
--enable-newline-is-anycrlf
- which causes PCRE2 to recognize any of the three sequences CR, LF, or
+ which causes PCRE2 to recognize any of the three sequences CR, LF, or
CRLF as indicating a line ending. A fifth option, specified by
--enable-newline-is-any
- causes PCRE2 to recognize any Unicode newline sequence. The Unicode
+ causes PCRE2 to recognize any Unicode newline sequence. The Unicode
newline sequences are the three just mentioned, plus the single charac-
ters VT (vertical tab, U+000B), FF (form feed, U+000C), NEL (next line,
- U+0085), LS (line separator, U+2028), and PS (paragraph separator,
+ U+0085), LS (line separator, U+2028), and PS (paragraph separator,
U+2029). The final option is
--enable-newline-is-nul
- which causes NUL (binary zero) to be set as the default line-ending
+ which causes NUL (binary zero) to be set as the default line-ending
character.
Whatever default line ending convention is selected when PCRE2 is built
- can be overridden by applications that use the library. At build time
+ can be overridden by applications that use the library. At build time
it is recommended to use the standard for your operating system.
WHAT \R MATCHES
- By default, the sequence \R in a pattern matches any Unicode newline
- sequence, independently of what has been selected as the line ending
+ By default, the sequence \R in a pattern matches any Unicode newline
+ sequence, independently of what has been selected as the line ending
sequence. If you specify
--enable-bsr-anycrlf
- the default is changed so that \R matches only CR, LF, or CRLF. What-
- ever is selected when PCRE2 is built can be overridden by applications
+ the default is changed so that \R matches only CR, LF, or CRLF. What-
+ ever is selected when PCRE2 is built can be overridden by applications
that use the library.
HANDLING VERY LARGE PATTERNS
- Within a compiled pattern, offset values are used to point from one
- part to another (for example, from an opening parenthesis to an alter-
- nation metacharacter). By default, in the 8-bit and 16-bit libraries,
- two-byte values are used for these offsets, leading to a maximum size
+ Within a compiled pattern, offset values are used to point from one
+ part to another (for example, from an opening parenthesis to an alter-
+ nation metacharacter). By default, in the 8-bit and 16-bit libraries,
+ two-byte values are used for these offsets, leading to a maximum size
for a compiled pattern of around 64 thousand code units. This is suffi-
- cient to handle all but the most gigantic patterns. Nevertheless, some
+ cient to handle all but the most gigantic patterns. Nevertheless, some
people do want to process truly enormous patterns, so it is possible to
- compile PCRE2 to use three-byte or four-byte offsets by adding a set-
+ compile PCRE2 to use three-byte or four-byte offsets by adding a set-
ting such as
--with-link-size=3
- to the configure command. The value given must be 2, 3, or 4. For the
- 16-bit library, a value of 3 is rounded up to 4. In these libraries,
- using longer offsets slows down the operation of PCRE2 because it has
- to load additional data when handling them. For the 32-bit library the
- value is always 4 and cannot be overridden; the value of --with-link-
+ to the configure command. The value given must be 2, 3, or 4. For the
+ 16-bit library, a value of 3 is rounded up to 4. In these libraries,
+ using longer offsets slows down the operation of PCRE2 because it has
+ to load additional data when handling them. For the 32-bit library the
+ value is always 4 and cannot be overridden; the value of --with-link-
size is ignored.
LIMITING PCRE2 RESOURCE USAGE
The pcre2_match() function increments a counter each time it goes round
- its main loop. Putting a limit on this counter controls the amount of
- computing resource used by a single call to pcre2_match(). The limit
+ its main loop. Putting a limit on this counter controls the amount of
+ computing resource used by a single call to pcre2_match(). The limit
can be changed at run time, as described in the pcre2api documentation.
- The default is 10 million, but this can be changed by adding a setting
+ The default is 10 million, but this can be changed by adding a setting
such as
--with-match-limit=500000
- to the configure command. This setting also applies to the
- pcre2_dfa_match() matching function, and to JIT matching (though the
+ to the configure command. This setting also applies to the
+ pcre2_dfa_match() matching function, and to JIT matching (though the
counting is done differently).
- The pcre2_match() function uses heap memory to record backtracking
- points. The more nested backtracking points there are (that is, the
- deeper the search tree), the more memory is needed. There is an upper
- limit, specified in kibibytes (units of 1024 bytes). This limit can be
- changed at run time, as described in the pcre2api documentation. The
- default limit (in effect unlimited) is 20 million. You can change this
+ The pcre2_match() function uses heap memory to record backtracking
+ points. The more nested backtracking points there are (that is, the
+ deeper the search tree), the more memory is needed. There is an upper
+ limit, specified in kibibytes (units of 1024 bytes). This limit can be
+ changed at run time, as described in the pcre2api documentation. The
+ default limit (in effect unlimited) is 20 million. You can change this
by a setting such as
--with-heap-limit=500
- which limits the amount of heap to 500 KiB. This limit applies only to
+ which limits the amount of heap to 500 KiB. This limit applies only to
interpretive matching in pcre2_match() and pcre2_dfa_match(), which may
- also use the heap for internal workspace when processing complicated
- patterns. This limit does not apply when JIT (which has its own memory
+ also use the heap for internal workspace when processing complicated
+ patterns. This limit does not apply when JIT (which has its own memory
arrangements) is used.
- You can also explicitly limit the depth of nested backtracking in the
+ You can also explicitly limit the depth of nested backtracking in the
pcre2_match() interpreter. This limit defaults to the value that is set
- for --with-match-limit. You can set a lower default limit by adding,
+ for --with-match-limit. You can set a lower default limit by adding,
for example,
--with-match-limit-depth=10000
- to the configure command. This value can be overridden at run time.
- This depth limit indirectly limits the amount of heap memory that is
- used, but because the size of each backtracking "frame" depends on the
- number of capturing parentheses in a pattern, the amount of heap that
- is used before the limit is reached varies from pattern to pattern.
+ to the configure command. This value can be overridden at run time.
+ This depth limit indirectly limits the amount of heap memory that is
+ used, but because the size of each backtracking "frame" depends on the
+ number of capturing parentheses in a pattern, the amount of heap that
+ is used before the limit is reached varies from pattern to pattern.
This limit was more useful in versions before 10.30, where function re-
cursion was used for backtracking.
As well as applying to pcre2_match(), the depth limit also controls the
- depth of recursive function calls in pcre2_dfa_match(). These are used
- for lookaround assertions, atomic groups, and recursion within pat-
+ depth of recursive function calls in pcre2_dfa_match(). These are used
+ for lookaround assertions, atomic groups, and recursion within pat-
terns. The limit does not apply to JIT matching.
LIMITING VARIABLE-LENGTH LOOKBEHIND ASSERTIONS
- Lookbehind assertions in which one or more branches can match a vari-
- able number of characters are supported only if there is a maximum
- matching length for each top-level branch. There is a limit to this
- maximum that defaults to 255 characters. You can alter this default by
+ Lookbehind assertions in which one or more branches can match a vari-
+ able number of characters are supported only if there is a maximum
+ matching length for each top-level branch. There is a limit to this
+ maximum that defaults to 255 characters. You can alter this default by
a setting such as
--with-max-varlookbehind=100
The limit can be changed at runtime by calling pcre2_set_max_varlookbe-
- hind(). Lookbehind assertions in which every branch matches a fixed
+ hind(). Lookbehind assertions in which every branch matches a fixed
number of characters (not necessarily all the same) are not constrained
by this limit.
@@ -4623,58 +4748,66 @@ CREATING CHARACTER TABLES AT BUILD TIME
PCRE2 uses fixed tables for processing characters whose code points are
less than 256. By default, PCRE2 is built with a set of tables that are
- distributed in the file src/pcre2_chartables.c.dist. These tables are
+ distributed in the file src/pcre2_chartables.c.dist. These tables are
for ASCII codes only. If you add
--enable-rebuild-chartables
- to the configure command, the distributed tables are no longer used.
+ to the configure command, the distributed tables are no longer used.
Instead, a program called pcre2_dftables is compiled and run. This out-
puts the source for new set of tables, created in the default locale of
- your C run-time system. This method of replacing the tables does not
+ your C run-time system. This method of replacing the tables does not
work if you are cross compiling, because pcre2_dftables needs to be run
on the local host and therefore not compiled with the cross compiler.
If you need to create alternative tables when cross compiling, you will
- have to do so "by hand". There may also be other reasons for creating
- tables manually. To cause pcre2_dftables to be built on the local
+ have to do so "by hand". There may also be other reasons for creating
+ tables manually. To cause pcre2_dftables to be built on the local
host, run a normal compiling command, and then run the program with the
output file as its argument, for example:
cc src/pcre2_dftables.c -o pcre2_dftables
./pcre2_dftables src/pcre2_chartables.c
- This builds the tables in the default locale of the local host. If you
+ This builds the tables in the default locale of the local host. If you
want to specify a locale, you must use the -L option:
LC_ALL=fr_FR ./pcre2_dftables -L src/pcre2_chartables.c
You can also specify -b (with or without -L). This causes the tables to
- be written in binary instead of as source code. A set of binary tables
- can be loaded into memory by an application and passed to pcre2_com-
+ be written in binary instead of as source code. A set of binary tables
+ can be loaded into memory by an application and passed to pcre2_com-
pile() in the same way as tables created by calling pcre2_maketables().
- The tables are just a string of bytes, independent of hardware charac-
- teristics such as endianness. This means they can be bundled with an
- application that runs in different environments, to ensure consistent
+ The tables are just a string of bytes, independent of hardware charac-
+ teristics such as endianness. This means they can be bundled with an
+ application that runs in different environments, to ensure consistent
behaviour.
USING EBCDIC CODE
- PCRE2 assumes by default that it will run in an environment where the
- character code is ASCII or Unicode, which is a superset of ASCII. This
+ PCRE2 assumes by default that it will run in an environment where the
+ character code is ASCII or Unicode, which is a superset of ASCII. This
is the case for most computer operating systems. PCRE2 can, however, be
compiled to run in an 8-bit EBCDIC environment by adding
--enable-ebcdic --disable-unicode
- to the configure command. This setting implies --enable-rebuild-charta-
- bles. You should only use it if you know that you are in an EBCDIC en-
- vironment (for example, an IBM mainframe operating system).
+ to the configure command. You should only use it if you know that you
+ are in an EBCDIC environment (for example, an IBM mainframe operating
+ system).
+
+ This setting implies --enable-rebuild-chartables, in order to ensure
+ that you have the correct default character tables for your system's
+ codepage. There is an exception when you set --enable-ebcdic-ignoring-
+ compiler (see below), which allows using a default set of EBCDIC 1047
+ character tables rather than forcing use of --enable-rebuild-charta-
+ bles.
- It is not possible to support both EBCDIC and UTF-8 codes in the same
- version of the library. Consequently, --enable-unicode and --enable-
- ebcdic are mutually exclusive.
+ It is not supported to enable both EBCDIC input and either ASCII or
+ UTF-8/16/32 in the same build of the library. When PCRE2 is built with
+ EBCDIC support, it always operates in EBCDIC, and consequently --en-
+ able-unicode and --enable-ebcdic are mutually exclusive.
The EBCDIC character that corresponds to an ASCII LF is assumed to have
the value 0x15 by default. However, in some EBCDIC environments, 0x25
@@ -4682,15 +4815,28 @@ USING EBCDIC CODE
--enable-ebcdic-nl25
- as well as, or instead of, --enable-ebcdic. The EBCDIC character for CR
- has the same value as in ASCII, namely, 0x0d. Whichever of 0x15 and
- 0x25 is not chosen as LF is made to correspond to the Unicode NEL char-
- acter (which, in Unicode, is 0x85).
+ (which implies --enable-ebcdic). The EBCDIC character for CR has the
+ same value as in ASCII, namely, 0x0d. Whichever of 0x15 and 0x25 is not
+ chosen as LF is made to correspond to the Unicode NEL character (which,
+ in Unicode, is 0x85).
The options that select newline behaviour, such as --enable-newline-is-
cr, and equivalent run-time options, refer to these character values in
an EBCDIC environment.
+ On systems requiring an EBCDIC build of PCRE2, the compiler should be
+ set to use the correct codepage, so that C character literals such as
+ 'z' use the correct numeric value for whichever EBCDIC codpage is in
+ use. (PCRE2 cannot support multiple EBCDIC codepages dynamically.) How-
+ ever, if this not possible, then you can use
+
+ --enable-ebcdic-ignoring-compiler
+
+ in order to disregard the compiler's codepage, and instead force PCRE2
+ to use numeric constants corresponding to the EBCDIC 1047 codepage in-
+ stead. This can be used to build (or test) EBCDIC support on an
+ ASCII/UTF-8 system such as Linux.
+
PCRE2GREP SUPPORT FOR EXTERNAL SCRIPTS
@@ -4768,7 +4914,7 @@ PCRE2TEST OPTION FOR LIBREADLINE SUPPORT
If your environment has not been set up so that an appropriate library
is automatically included, you may need to add something like
- LIBS="-ncurses"
+ LIBS="-lncurses"
immediately before the configure command.
@@ -4919,11 +5065,11 @@ AUTHOR
REVISION
- Last updated: 16 April 2024
+ Last updated: 17 October 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 16 April 2024 PCRE2BUILD(3)
+PCRE2 10.47 17 October 2025 PCRE2BUILD(3)
------------------------------------------------------------------------------
@@ -5352,11 +5498,11 @@ AUTHOR
REVISION
- Last updated: 19 January 2024
+ Last updated: 26 February 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 19 January 2024 PCRE2CALLOUT(3)
+PCRE2 10.47 26 February 2025 PCRE2CALLOUT(3)
------------------------------------------------------------------------------
@@ -5397,7 +5543,12 @@ DIFFERENCES BETWEEN PCRE2 AND PERL
4. If a braced quantifier such as {1,2} appears where there is nothing
to repeat (for example, at the start of a branch), PCRE2 raises an er-
- ror whereas Perl treats the quantifier characters as literal.
+ ror whereas Perl treats the quantifier characters as literal. When a
+ braced quantifier (...){min,max} has min > max, Perl treats it as an
+ item which fails to match any portion of the subject (as no number of
+ repetitions can meet the condition), and additionally issues a warning
+ when in warning mode. PCRE2 has no warning features, so it gives an er-
+ ror in this case.
5. Capture groups that occur inside negative lookaround assertions are
counted, but their entries in the offsets vector are set only when a
@@ -5546,7 +5697,7 @@ DIFFERENCES BETWEEN PCRE2 AND PERL
(h) The partial matching facility is PCRE2-specific.
- (i) The alternative matching function (pcre2_dfa_match() matches in a
+ (i) The alternative matching function (pcre2_dfa_match()) matches in a
different way and is not Perl-compatible.
(j) PCRE2 recognizes some special sequences such as (*CR) or (*NO_JIT)
@@ -5567,12 +5718,12 @@ DIFFERENCES BETWEEN PCRE2 AND PERL
(m) The scan substring assertion (syntax (*scs:(n)...)) is a PCRE2 ex-
tension that is not available in Perl.
- 20. Perl has different limits than PCRE2. See the pcre2limit documenta-
- tion for details. Perl went with 5.10 from recursion to iteration keep-
- ing the intermediate matches on the heap, which is ~10% slower but does
- not fall into any stack-overflow limit. PCRE2 made a similar change at
- release 10.30, and also has many build-time and run-time customizable
- limits.
+ 20. Perl has different limits than PCRE2. See the pcre2limits documen-
+ tation for details. Perl went with 5.10 from recursion to iteration
+ keeping the intermediate matches on the heap, which is ~10% slower but
+ does not fall into any stack-overflow limit. PCRE2 made a similar
+ change at release 10.30, and also has many build-time and run-time cus-
+ tomizable limits.
21. Unlike Perl, PCRE2 doesn't have character set modifiers and spe-
cially no way to set characters by context just like Perl's "/d". A
@@ -5587,7 +5738,7 @@ DIFFERENCES BETWEEN PCRE2 AND PERL
23. Both PCRE2 and Perl error when \x{ escapes are invalid, but Perl
tries to recover and prints a warning if the problem was that an in-
- valid hexadecimal digit was found, since PCRE2 doesn't have warnings it
+ valid hexadecimal digit was found. Since PCRE2 doesn't have warnings it
returns an error instead. Additionally, Perl accepts \x{} and gener-
ates NUL unlike PCRE2.
@@ -5606,11 +5757,11 @@ AUTHOR
REVISION
- Last updated: 02 October 2024
+ Last updated: 02 June 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 02 October 2024 PCRE2COMPAT(3)
+PCRE2 10.47 02 June 2025 PCRE2COMPAT(3)
------------------------------------------------------------------------------
@@ -6065,7 +6216,7 @@ REVISION
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 22 August 2024 PCRE2JIT(3)
+PCRE2 10.47 22 August 2024 PCRE2JIT(3)
------------------------------------------------------------------------------
@@ -6120,18 +6271,20 @@ SIZE AND OTHER LIMITATIONS
is set to 250. An application can change this limit by calling
pcre2_set_parens_nest_limit() to set the limit in a compile context.
- The maximum length of name for a named capture group is 32 code units,
- and the maximum number of such groups is 10000.
+ The maximum length of the name for a named capture group as well as the
+ number of such groups is configurable at build time. The maximum length
+ for the name defaults to 128 code units, and the maximum number of such
+ groups to 10000.
- The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or
- (*THEN) verb is 255 code units for the 8-bit library and 65535 code
+ The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or
+ (*THEN) verb is 255 code units for the 8-bit library and 65535 code
units for the 16-bit and 32-bit libraries.
- The maximum length of a string argument to a callout is the largest
+ The maximum length of a string argument to a callout is the largest
number a 32-bit unsigned integer can hold.
- The maximum amount of heap memory used for matching is controlled by
- the heap limit, which can be set in a pattern or in a match context.
+ The maximum amount of heap memory used for matching is controlled by
+ the heap limit, which can be set in a pattern or in a match context.
The default is a very large number, effectively unlimited.
@@ -6144,11 +6297,11 @@ AUTHOR
REVISION
- Last updated: 16 August 2023
+ Last updated: 03 September 2025
Copyright (c) 1997-2023 University of Cambridge.
-PCRE2 10.46 16 August 2023 PCRE2LIMITS(3)
+PCRE2 10.47 03 September 2025 PCRE2LIMITS(3)
------------------------------------------------------------------------------
@@ -6164,18 +6317,18 @@ PCRE2 MATCHING ALGORITHMS
This document describes the two different algorithms that are available
in PCRE2 for matching a compiled regular expression against a given
subject string. The "standard" algorithm is the one provided by the
- pcre2_match() function. This works in the same as Perl's matching func-
- tion, and provides a Perl-compatible matching operation. The just-in-
- time (JIT) optimization that is described in the pcre2jit documentation
- is compatible with this function.
+ pcre2_match() function. This works in the same way as Perl's matching
+ function, and provides a Perl-compatible matching operation. The just-
+ in-time (JIT) optimization that is described in the pcre2jit documenta-
+ tion is compatible with this function.
An alternative algorithm is provided by the pcre2_dfa_match() function;
it operates in a different way, and is not Perl-compatible. This alter-
- native has advantages and disadvantages compared with the standard al-
+ native has advantages and disadvantages compared with the standard al-
gorithm, and these are described below.
When there is only one possible way in which a given subject string can
- match a pattern, the two algorithms give the same answer. A difference
+ match a pattern, the two algorithms give the same answer. A difference
arises, however, when there are multiple possibilities. For example, if
the anchored pattern
@@ -6192,114 +6345,114 @@ PCRE2 MATCHING ALGORITHMS
REGULAR EXPRESSIONS AS TREES
The set of strings that are matched by a regular expression can be rep-
- resented as a tree structure. An unlimited repetition in the pattern
- makes the tree of infinite size, but it is still a tree. Matching the
- pattern to a given subject string (from a given starting point) can be
- thought of as a search of the tree. There are two ways to search a
- tree: depth-first and breadth-first, and these correspond to the two
+ resented as a tree structure. An unlimited repetition in the pattern
+ makes the tree of infinite size, but it is still a tree. Matching the
+ pattern to a given subject string (from a given starting point) can be
+ thought of as a search of the tree. There are two ways to search a
+ tree: depth-first and breadth-first, and these correspond to the two
matching algorithms provided by PCRE2.
THE STANDARD MATCHING ALGORITHM
- In the terminology of Jeffrey Friedl's book "Mastering Regular Expres-
- sions", the standard algorithm is an "NFA algorithm". It conducts a
- depth-first search of the pattern tree. That is, it proceeds along a
+ In the terminology of Jeffrey Friedl's book "Mastering Regular Expres-
+ sions", the standard algorithm is an "NFA algorithm". It conducts a
+ depth-first search of the pattern tree. That is, it proceeds along a
single path through the tree, checking that the subject matches what is
- required. When there is a mismatch, the algorithm tries any alterna-
- tives at the current point, and if they all fail, it backs up to the
- previous branch point in the tree, and tries the next alternative
- branch at that level. This often involves backing up (moving to the
- left) in the subject string as well. The order in which repetition
- branches are tried is controlled by the greedy or ungreedy nature of
+ required. When there is a mismatch, the algorithm tries any alterna-
+ tives at the current point, and if they all fail, it backs up to the
+ previous branch point in the tree, and tries the next alternative
+ branch at that level. This often involves backing up (moving to the
+ left) in the subject string as well. The order in which repetition
+ branches are tried is controlled by the greedy or ungreedy nature of
the quantifier.
- If a leaf node is reached, a matching string has been found, and at
- that point the algorithm stops. Thus, if there is more than one possi-
- ble match, this algorithm returns the first one that it finds. Whether
- this is the shortest, the longest, or some intermediate length depends
+ If a leaf node is reached, a matching string has been found, and at
+ that point the algorithm stops. Thus, if there is more than one possi-
+ ble match, this algorithm returns the first one that it finds. Whether
+ this is the shortest, the longest, or some intermediate length depends
on the way the alternations and the greedy or ungreedy repetition quan-
tifiers are specified in the pattern.
- Because it ends up with a single path through the tree, it is rela-
- tively straightforward for this algorithm to keep track of the sub-
- strings that are matched by portions of the pattern in parentheses.
+ Because it ends up with a single path through the tree, it is rela-
+ tively straightforward for this algorithm to keep track of the sub-
+ strings that are matched by portions of the pattern in parentheses.
This provides support for capturing parentheses and backreferences.
THE ALTERNATIVE MATCHING ALGORITHM
- This algorithm conducts a breadth-first search of the tree. Starting
- from the first matching point in the subject, it scans the subject
+ This algorithm conducts a breadth-first search of the tree. Starting
+ from the first matching point in the subject, it scans the subject
string from left to right, once, character by character, and as it does
- this, it remembers all the paths through the tree that represent valid
- matches. In Friedl's terminology, this is a kind of "DFA algorithm",
- though it is not implemented as a traditional finite state machine (it
+ this, it remembers all the paths through the tree that represent valid
+ matches. In Friedl's terminology, this is a kind of "DFA algorithm",
+ though it is not implemented as a traditional finite state machine (it
keeps multiple states active simultaneously).
- Although the general principle of this matching algorithm is that it
- scans the subject string only once, without backtracking, there is one
- exception: when a lookaround assertion is encountered, the characters
- following or preceding the current point have to be independently in-
+ Although the general principle of this matching algorithm is that it
+ scans the subject string only once, without backtracking, there is one
+ exception: when a lookaround assertion is encountered, the characters
+ following or preceding the current point have to be independently in-
spected.
- The scan continues until either the end of the subject is reached, or
- there are no more unterminated paths. At this point, terminated paths
- represent the different matching possibilities (if there are none, the
- match has failed). Thus, if there is more than one possible match,
- this algorithm finds all of them, and in particular, it finds the
- longest. The matches are returned in the output vector in decreasing
- order of length. There is an option to stop the algorithm after the
+ The scan continues until either the end of the subject is reached, or
+ there are no more unterminated paths. At this point, terminated paths
+ represent the different matching possibilities (if there are none, the
+ match has failed). Thus, if there is more than one possible match,
+ this algorithm finds all of them, and in particular, it finds the
+ longest. The matches are returned in the output vector in decreasing
+ order of length. There is an option to stop the algorithm after the
first match (which is necessarily the shortest) is found.
- Note that the size of vector needed to contain all the results depends
- on the number of simultaneous matches, not on the number of capturing
- parentheses in the pattern. Using pcre2_match_data_create_from_pat-
- tern() to create the match data block is therefore not advisable when
+ Note that the size of vector needed to contain all the results depends
+ on the number of simultaneous matches, not on the number of capturing
+ parentheses in the pattern. Using pcre2_match_data_create_from_pat-
+ tern() to create the match data block is therefore not advisable when
doing DFA matching.
- Note also that all the matches that are found start at the same point
+ Note also that all the matches that are found start at the same point
in the subject. If the pattern
cat(er(pillar)?)?
- is matched against the string "the caterpillar catchment", the result
- is the three strings "caterpillar", "cater", and "cat" that start at
- the fifth character of the subject. The algorithm does not automati-
+ is matched against the string "the caterpillar catchment", the result
+ is the three strings "caterpillar", "cater", and "cat" that start at
+ the fifth character of the subject. The algorithm does not automati-
cally move on to find matches that start at later positions.
PCRE2's "auto-possessification" optimization usually applies to charac-
- ter repeats at the end of a pattern (as well as internally). For exam-
+ ter repeats at the end of a pattern (as well as internally). For exam-
ple, the pattern "a\d+" is compiled as if it were "a\d++" because there
- is no point even considering the possibility of backtracking into the
- repeated digits. For DFA matching, this means that only one possible
- match is found. If you really do want multiple matches in such cases,
- either use an ungreedy repeat ("a\d+?") or set the PCRE2_NO_AUTO_POS-
+ is no point even considering the possibility of backtracking into the
+ repeated digits. For DFA matching, this means that only one possible
+ match is found. If you really do want multiple matches in such cases,
+ either use an ungreedy repeat ("a\d+?") or set the PCRE2_NO_AUTO_POS-
SESS option when compiling.
- There are a number of features of PCRE2 regular expressions that are
- not supported or behave differently in the alternative matching func-
+ There are a number of features of PCRE2 regular expressions that are
+ not supported or behave differently in the alternative matching func-
tion. Those that are not supported cause an error if encountered.
- 1. Because the algorithm finds all possible matches, the greedy or un-
- greedy nature of repetition quantifiers is not relevant (though it may
- affect auto-possessification, as just described). During matching,
- greedy and ungreedy quantifiers are treated in exactly the same way.
+ 1. Because the algorithm finds all possible matches, the greedy or un-
+ greedy nature of repetition quantifiers is not relevant (though it may
+ affect auto-possessification, as just described). During matching,
+ greedy and ungreedy quantifiers are treated in exactly the same way.
However, possessive quantifiers can make a difference when what follows
- could also match what is quantified, for example in a pattern like
+ could also match what is quantified, for example in a pattern like
this:
^a++\w!
- This pattern matches "aaab!" but not "aaa!", which would be matched by
- a non-possessive quantifier. Similarly, if an atomic group is present,
- it is matched as if it were a standalone pattern at the current point,
- and the longest match is then "locked in" for the rest of the overall
+ This pattern matches "aaab!" but not "aaa!", which would be matched by
+ a non-possessive quantifier. Similarly, if an atomic group is present,
+ it is matched as if it were a standalone pattern at the current point,
+ and the longest match is then "locked in" for the rest of the overall
pattern.
2. When dealing with multiple paths through the tree simultaneously, it
- is not straightforward to keep track of captured substrings for the
- different matching possibilities, and PCRE2's implementation of this
+ is not straightforward to keep track of captured substrings for the
+ different matching possibilities, and PCRE2's implementation of this
algorithm does not attempt to do this. This means that no captured sub-
strings are available.
@@ -6308,7 +6461,7 @@ THE ALTERNATIVE MATCHING ALGORITHM
(a) Backreferences;
- (b) Conditional expressions that use a backreference as the condition
+ (b) Conditional expressions that use a backreference as the condition
or test for a specific group recursion;
(c) Script runs;
@@ -6316,36 +6469,36 @@ THE ALTERNATIVE MATCHING ALGORITHM
(d) Scan substring assertions.
4. Because many paths through the tree may be active, the \K escape se-
- quence, which resets the start of the match when encountered (but may
+ quence, which resets the start of the match when encountered (but may
be on some paths and not on others), is not supported.
- 5. Callouts are supported, but the value of the capture_top field is
+ 5. Callouts are supported, but the value of the capture_top field is
always 1, and the value of the capture_last field is always 0.
- 6. The \C escape sequence, which (in the standard algorithm) always
+ 6. The \C escape sequence, which (in the standard algorithm) always
matches a single code unit, even in a UTF mode, is not supported in UTF
- modes because the alternative algorithm moves through the subject
- string one character (not code unit) at a time, for all active paths
+ modes because the alternative algorithm moves through the subject
+ string one character (not code unit) at a time, for all active paths
through the tree.
- 7. Except for (*FAIL), the backtracking control verbs such as (*PRUNE)
- are not supported. (*FAIL) is supported, and behaves like a failing
+ 7. Except for (*FAIL), the backtracking control verbs such as (*PRUNE)
+ are not supported. (*FAIL) is supported, and behaves like a failing
negative assertion.
- 8. The PCRE2_MATCH_INVALID_UTF option for pcre2_compile() is not sup-
+ 8. The PCRE2_MATCH_INVALID_UTF option for pcre2_compile() is not sup-
ported by pcre2_dfa_match().
ADVANTAGES OF THE ALTERNATIVE ALGORITHM
- The main advantage of the alternative algorithm is that all possible
+ The main advantage of the alternative algorithm is that all possible
matches (at a single point in the subject) are automatically found, and
- in particular, the longest match is found. To find more than one match
- at the same point using the standard algorithm, you have to do kludgy
+ in particular, the longest match is found. To find more than one match
+ at the same point using the standard algorithm, you have to do kludgy
things with callouts.
- Partial matching is possible with this algorithm, though it has some
- limitations. The pcre2partial documentation gives details of partial
+ Partial matching is possible with this algorithm, though it has some
+ limitations. The pcre2partial documentation gives details of partial
matching and discusses multi-segment matching.
@@ -6353,8 +6506,8 @@ DISADVANTAGES OF THE ALTERNATIVE ALGORITHM
The alternative algorithm suffers from a number of disadvantages:
- 1. It is substantially slower than the standard algorithm. This is
- partly because it has to search for all possible matches, but is also
+ 1. It is substantially slower than the standard algorithm. This is
+ partly because it has to search for all possible matches, but is also
because it is less susceptible to optimization.
2. Capturing parentheses and other features such as backreferences that
@@ -6377,11 +6530,11 @@ AUTHOR
REVISION
- Last updated: 30 August 2024
+ Last updated: 22 February 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 30 August 2024 PCRE2MATCHING(3)
+PCRE2 10.47 22 February 2025 PCRE2MATCHING(3)
------------------------------------------------------------------------------
@@ -6764,7 +6917,7 @@ REVISION
Copyright (c) 1997-2019 University of Cambridge.
-PCRE2 10.46 27 November 2024 PCRE2PARTIAL(3)
+PCRE2 10.47 27 November 2024 PCRE2PARTIAL(3)
------------------------------------------------------------------------------
@@ -6892,32 +7045,32 @@ SPECIAL START-OF-PATTERN ITEMS
If a pattern starts with (*NO_DOTSTAR_ANCHOR), it has the same effect
as setting the PCRE2_NO_DOTSTAR_ANCHOR option, or calling pcre2_set_op-
- timize() with a PCRE2_DOTSTAR_ANCHOR_OFF directive. This disables opti-
- mizations that apply to patterns whose top-level branches all start
- with .* (match any number of arbitrary characters). For more details,
+ timize() with a PCRE2_DOTSTAR_ANCHOR_OFF directive. This disables op-
+ timizations that apply to patterns whose top-level branches all start
+ with .* (match any number of arbitrary characters). For more details,
see the pcre2api documentation.
Disabling JIT compilation
- If a pattern that starts with (*NO_JIT) is successfully compiled, an
- attempt by the application to apply the JIT optimization by calling
+ If a pattern that starts with (*NO_JIT) is successfully compiled, an
+ attempt by the application to apply the JIT optimization by calling
pcre2_jit_compile() is ignored.
Setting match resource limits
The pcre2_match() function contains a counter that is incremented every
time it goes round its main loop. The caller of pcre2_match() can set a
- limit on this counter, which therefore limits the amount of computing
+ limit on this counter, which therefore limits the amount of computing
resource used for a match. The maximum depth of nested backtracking can
- also be limited; this indirectly restricts the amount of heap memory
- that is used, but there is also an explicit memory limit that can be
+ also be limited; this indirectly restricts the amount of heap memory
+ that is used, but there is also an explicit memory limit that can be
set.
- These facilities are provided to catch runaway matches that are pro-
- voked by patterns with huge matching trees. A common example is a pat-
- tern with nested unlimited repeats applied to a long string that does
- not match. When one of these limits is reached, pcre2_match() gives an
- error return. The limits can also be set by items at the start of the
+ These facilities are provided to catch runaway matches that are pro-
+ voked by patterns with huge matching trees. A common example is a pat-
+ tern with nested unlimited repeats applied to a long string that does
+ not match. When one of these limits is reached, pcre2_match() gives an
+ error return. The limits can also be set by items at the start of the
pattern of the form
(*LIMIT_HEAP=d)
@@ -6925,35 +7078,35 @@ SPECIAL START-OF-PATTERN ITEMS
(*LIMIT_DEPTH=d)
where d is any number of decimal digits. However, the value of the set-
- ting must be less than the value set (or defaulted) by the caller of
- pcre2_match() for it to have any effect. In other words, the pattern
- writer can lower the limits set by the programmer, but not raise them.
- If there is more than one setting of one of these limits, the lower
- value is used. The heap limit is specified in kibibytes (units of 1024
+ ting must be less than the value set (or defaulted) by the caller of
+ pcre2_match() for it to have any effect. In other words, the pattern
+ writer can lower the limits set by the programmer, but not raise them.
+ If there is more than one setting of one of these limits, the lower
+ value is used. The heap limit is specified in kibibytes (units of 1024
bytes).
- Prior to release 10.30, LIMIT_DEPTH was called LIMIT_RECURSION. This
+ Prior to release 10.30, LIMIT_DEPTH was called LIMIT_RECURSION. This
name is still recognized for backwards compatibility.
The heap limit applies only when the pcre2_match() or pcre2_dfa_match()
interpreters are used for matching. It does not apply to JIT. The match
- limit is used (but in a different way) when JIT is being used, or when
+ limit is used (but in a different way) when JIT is being used, or when
pcre2_dfa_match() is called, to limit computing resource usage by those
- matching functions. The depth limit is ignored by JIT but is relevant
- for DFA matching, which uses function recursion for recursions within
- the pattern and for lookaround assertions and atomic groups. In this
+ matching functions. The depth limit is ignored by JIT but is relevant
+ for DFA matching, which uses function recursion for recursions within
+ the pattern and for lookaround assertions and atomic groups. In this
case, the depth limit controls the depth of such recursion.
Newline conventions
- PCRE2 supports six different conventions for indicating line breaks in
- strings: a single CR (carriage return) character, a single LF (line-
+ PCRE2 supports six different conventions for indicating line breaks in
+ strings: a single CR (carriage return) character, a single LF (line-
feed) character, the two-character sequence CRLF, any of the three pre-
- ceding, any Unicode newline sequence, or the NUL character (binary
- zero). The pcre2api page has further discussion about newlines, and
+ ceding, any Unicode newline sequence, or the NUL character (binary
+ zero). The pcre2api page has further discussion about newlines, and
shows how to set the newline convention when calling pcre2_compile().
- It is also possible to specify a newline convention by starting a pat-
+ It is also possible to specify a newline convention by starting a pat-
tern string with one of the following sequences:
(*CR) carriage return
@@ -6973,56 +7126,56 @@ SPECIAL START-OF-PATTERN ITEMS
no longer a newline. If more than one of these settings is present, the
last one is used.
- The newline convention affects where the circumflex and dollar asser-
+ The newline convention affects where the circumflex and dollar asser-
tions are true. It also affects the interpretation of the dot metachar-
- acter when PCRE2_DOTALL is not set, and the behaviour of \N when not
- followed by an opening brace. However, it does not affect what the \R
- escape sequence matches. By default, this is any Unicode newline se-
- quence, for Perl compatibility. However, this can be changed; see the
+ acter when PCRE2_DOTALL is not set, and the behaviour of \N when not
+ followed by an opening brace. However, it does not affect what the \R
+ escape sequence matches. By default, this is any Unicode newline se-
+ quence, for Perl compatibility. However, this can be changed; see the
next section and the description of \R in the section entitled "Newline
- sequences" below. A change of \R setting can be combined with a change
+ sequences" below. A change of \R setting can be combined with a change
of newline convention.
Specifying what \R matches
It is possible to restrict \R to match only CR, LF, or CRLF (instead of
- the complete set of Unicode line endings) by setting the option
- PCRE2_BSR_ANYCRLF at compile time. This effect can also be achieved by
- starting a pattern with (*BSR_ANYCRLF). For completeness, (*BSR_UNI-
+ the complete set of Unicode line endings) by setting the option
+ PCRE2_BSR_ANYCRLF at compile time. This effect can also be achieved by
+ starting a pattern with (*BSR_ANYCRLF). For completeness, (*BSR_UNI-
CODE) is also recognized, corresponding to PCRE2_BSR_UNICODE.
CHARACTERS AND METACHARACTERS
- A regular expression is a pattern that is matched against a subject
- string from left to right. Most characters stand for themselves in a
- pattern, and match the corresponding characters in the subject. As a
+ A regular expression is a pattern that is matched against a subject
+ string from left to right. Most characters stand for themselves in a
+ pattern, and match the corresponding characters in the subject. As a
trivial example, the pattern
The quick brown fox
matches a portion of a subject string that is identical to itself. When
- caseless matching is specified (the PCRE2_CASELESS option or (?i)
- within the pattern), letters are matched independently of case. Note
- that there are two ASCII characters, K and S, that, in addition to
- their lower case ASCII equivalents, are case-equivalent with Unicode
- U+212A (Kelvin sign) and U+017F (long S) respectively when either
+ caseless matching is specified (the PCRE2_CASELESS option or (?i)
+ within the pattern), letters are matched independently of case. Note
+ that there are two ASCII characters, K and S, that, in addition to
+ their lower case ASCII equivalents, are case-equivalent with Unicode
+ U+212A (Kelvin sign) and U+017F (long S) respectively when either
PCRE2_UTF or PCRE2_UCP is set, unless the PCRE2_EXTRA_CASELESS_RESTRICT
- option is in force (either passed to pcre2_compile() or set by (*CASE-
- LESS_RESTRICT) or (?r) within the pattern). If the PCRE2_EXTRA_TURK-
- ISH_CASING option is in force (either passed to pcre2_compile() or set
- by (*TURKISH_CASING) within the pattern), then the 'i' letters are
+ option is in force (either passed to pcre2_compile() or set by (*CASE-
+ LESS_RESTRICT) or (?r) within the pattern). If the PCRE2_EXTRA_TURK-
+ ISH_CASING option is in force (either passed to pcre2_compile() or set
+ by (*TURKISH_CASING) within the pattern), then the 'i' letters are
matched according to Turkish and Azeri languages.
The power of regular expressions comes from the ability to include wild
cards, character classes, alternatives, and repetitions in the pattern.
These are encoded in the pattern by the use of metacharacters, which do
- not stand for themselves but instead are interpreted in some special
+ not stand for themselves but instead are interpreted in some special
way.
- There are two different sets of metacharacters: those that are recog-
- nized anywhere in the pattern except within square brackets, and those
- that are recognized within square brackets. Outside square brackets,
+ There are two different sets of metacharacters: those that are recog-
+ nized anywhere in the pattern except within square brackets, and those
+ that are recognized within square brackets. Outside square brackets,
the metacharacters are as follows:
\ general escape character with several uses
@@ -7038,16 +7191,16 @@ CHARACTERS AND METACHARACTERS
? 0 or 1 quantifier; also quantifier minimizer
{ potential start of min/max quantifier
- Brace characters { and } are also used to enclose data for construc-
- tions such as \g{2} or \k{name}. In almost all uses of braces, space
+ Brace characters { and } are also used to enclose data for construc-
+ tions such as \g{2} or \k{name}. In almost all uses of braces, space
and/or horizontal tab characters that follow { or precede } are allowed
- and are ignored. In the case of quantifiers, they may also appear be-
- fore or after the comma. The exception to this is \u{...} which is an
- ECMAScript compatibility feature that is recognized only when the
- PCRE2_EXTRA_ALT_BSUX option is set. ECMAScript does not ignore such
+ and are ignored. In the case of quantifiers, they may also appear be-
+ fore or after the comma. The exception to this is \u{...} which is an
+ ECMAScript compatibility feature that is recognized only when the
+ PCRE2_EXTRA_ALT_BSUX option is set. ECMAScript does not ignore such
white space; it causes the item to be interpreted as literal.
- Part of a pattern that is in square brackets is called a "character
+ Part of a pattern that is in square brackets is called a "character
class". In a character class the only metacharacters are:
\ general escape character
@@ -7056,16 +7209,16 @@ CHARACTERS AND METACHARACTERS
[ POSIX character class (if followed by POSIX syntax)
] terminates the character class
- If a pattern is compiled with the PCRE2_EXTENDED option, most white
+ If a pattern is compiled with the PCRE2_EXTENDED option, most white
space in the pattern, other than in a character class, within a \Q...\E
- sequence, or between a # outside a character class and the next new-
- line, inclusive, is ignored. An escaping backslash can be used to in-
- clude a white space or a # character as part of the pattern. If the
- PCRE2_EXTENDED_MORE option is set, the same applies, but in addition
- unescaped space and horizontal tab characters are ignored inside a
- character class. Note: only these two characters are ignored, not the
- full set of pattern white space characters that are ignored outside a
- character class. Option settings can be changed within a pattern; see
+ sequence, or between a # outside a character class and the next new-
+ line, inclusive, is ignored. An escaping backslash can be used to in-
+ clude a white space or a # character as part of the pattern. If the
+ PCRE2_EXTENDED_MORE option is set, the same applies, but in addition
+ unescaped space and horizontal tab characters are ignored inside a
+ character class. Note: only these two characters are ignored, not the
+ full set of pattern white space characters that are ignored outside a
+ character class. Option settings can be changed within a pattern; see
the section entitled "Internal Option Setting" below.
The following sections describe the use of each of the metacharacters.
@@ -7074,30 +7227,30 @@ CHARACTERS AND METACHARACTERS
BACKSLASH
The backslash character has several uses. Firstly, if it is followed by
- a character that is not a digit or a letter, it takes away any special
- meaning that character may have. This use of backslash as an escape
+ a character that is not a digit or a letter, it takes away any special
+ meaning that character may have. This use of backslash as an escape
character applies both inside and outside character classes.
- For example, if you want to match a * character, you must write \* in
- the pattern. This escaping action applies whether or not the following
- character would otherwise be interpreted as a metacharacter, so it is
- always safe to precede a non-alphanumeric with backslash to specify
+ For example, if you want to match a * character, you must write \* in
+ the pattern. This escaping action applies whether or not the following
+ character would otherwise be interpreted as a metacharacter, so it is
+ always safe to precede a non-alphanumeric with backslash to specify
that it stands for itself. In particular, if you want to match a back-
slash, you write \\.
- Only ASCII digits and letters have any special meaning after a back-
+ Only ASCII digits and letters have any special meaning after a back-
slash. All other characters (in particular, those whose code points are
greater than 127) are treated as literals.
- If you want to treat all characters in a sequence as literals, you can
- do so by putting them between \Q and \E. Note that this includes white
- space even when the PCRE2_EXTENDED option is set so that most other
- white space is ignored. The behaviour is different from Perl in that $
+ If you want to treat all characters in a sequence as literals, you can
+ do so by putting them between \Q and \E. Note that this includes white
+ space even when the PCRE2_EXTENDED option is set so that most other
+ white space is ignored. The behaviour is different from Perl in that $
and @ are handled as literals in \Q...\E sequences in PCRE2, whereas in
- Perl, $ and @ cause variable interpolation. Also, Perl does "double-
- quotish backslash interpolation" on any backslashes between \Q and \E
- which, its documentation says, "may lead to confusing results". PCRE2
- treats a backslash between \Q and \E just like any other character.
+ Perl, $ and @ cause variable interpolation. Also, Perl does "double-
+ quotish backslash interpolation" on any backslashes between \Q and \E
+ which, its documentation says, "may lead to confusing results". PCRE2
+ treats a backslash between \Q and \E just like any other character.
Note the following examples:
Pattern PCRE2 matches Perl matches
@@ -7109,29 +7262,29 @@ BACKSLASH
\QA\B\E A\B A\B
\Q\\E \ \\E
- The \Q...\E sequence is recognized both inside and outside character
- classes. An isolated \E that is not preceded by \Q is ignored. If \Q
- is not followed by \E later in the pattern, the literal interpretation
- continues to the end of the pattern (that is, \E is assumed at the
- end). If the isolated \Q is inside a character class, this causes an
- error, because the character class is then not terminated by a closing
+ The \Q...\E sequence is recognized both inside and outside character
+ classes. An isolated \E that is not preceded by \Q is ignored. If \Q
+ is not followed by \E later in the pattern, the literal interpretation
+ continues to the end of the pattern (that is, \E is assumed at the
+ end). If the isolated \Q is inside a character class, this causes an
+ error, because the character class is then not terminated by a closing
square bracket.
- Another difference from Perl is that any appearance of \Q or \E inside
- what might otherwise be a quantifier causes PCRE2 not to recognize the
+ Another difference from Perl is that any appearance of \Q or \E inside
+ what might otherwise be a quantifier causes PCRE2 not to recognize the
sequence as a quantifier. Perl recognizes a quantifier if (redundantly)
- either of the numbers is inside \Q...\E, but not if the separating
- comma is. When not recognized as a quantifier a sequence such as
+ either of the numbers is inside \Q...\E, but not if the separating
+ comma is. When not recognized as a quantifier a sequence such as
{\Q1\E,2} is treated as the literal string "{1,2}".
Non-printing characters
A second use of backslash provides a way of encoding non-printing char-
- acters in patterns in a visible manner. There is no restriction on the
- appearance of non-printing characters in a pattern, but when a pattern
+ acters in patterns in a visible manner. There is no restriction on the
+ appearance of non-printing characters in a pattern, but when a pattern
is being prepared by text editing, it is often easier to use one of the
- following escape sequences instead of the binary character it repre-
- sents. In an ASCII or Unicode environment, these escapes are as fol-
+ following escape sequences instead of the binary character it repre-
+ sents. In an ASCII or Unicode environment, these escapes are as fol-
lows:
\a alarm, that is, the BEL character (hex 07)
@@ -7151,97 +7304,97 @@ BACKSLASH
A description of how back references work is given later, following the
discussion of parenthesized groups.
- By default, after \x that is not followed by {, one or two hexadecimal
+ By default, after \x that is not followed by {, one or two hexadecimal
digits are read (letters can be in upper or lower case). If the charac-
- ter that follows \x is neither { nor a hexadecimal digit, an error oc-
- curs. This is different from Perl's default behaviour, which generates
- a NUL character, but is in line with the behaviour of Perl's 'strict'
+ ter that follows \x is neither { nor a hexadecimal digit, an error oc-
+ curs. This is different from Perl's default behaviour, which generates
+ a NUL character, but is in line with the behaviour of Perl's 'strict'
mode in re.
- Any number of hexadecimal digits may appear between \x{ and }. If a
- character other than a hexadecimal digit appears between \x{ and }, or
+ Any number of hexadecimal digits may appear between \x{ and }. If a
+ character other than a hexadecimal digit appears between \x{ and }, or
if there is no terminating }, an error occurs.
Characters whose code points are less than 256 can be defined by either
of the two syntaxes for \x or by an octal sequence. There is no differ-
ence in the way they are handled. For example, \xdc is exactly the same
- as \x{dc} or \334. However, using the braced versions does make such
+ as \x{dc} or \334. However, using the braced versions does make such
sequences easier to read.
- Support is available for some ECMAScript (aka JavaScript) escape se-
+ Support is available for some ECMAScript (aka JavaScript) escape se-
quences via two compile-time options. If PCRE2_ALT_BSUX is set, the se-
- quence \x followed by { is not recognized. Only if \x is followed by
- two hexadecimal digits is it recognized as a character escape. Other-
- wise it is interpreted as a literal "x" character. In this mode, sup-
- port for code points greater than 256 is provided by \u, which must be
- followed by four hexadecimal digits; otherwise it is interpreted as a
+ quence \x followed by { is not recognized. Only if \x is followed by
+ two hexadecimal digits is it recognized as a character escape. Other-
+ wise it is interpreted as a literal "x" character. In this mode, sup-
+ port for code points greater than 256 is provided by \u, which must be
+ followed by four hexadecimal digits; otherwise it is interpreted as a
literal "u" character.
- PCRE2_EXTRA_ALT_BSUX has the same effect as PCRE2_ALT_BSUX and, in ad-
+ PCRE2_EXTRA_ALT_BSUX has the same effect as PCRE2_ALT_BSUX and, in ad-
dition, \u{hhh..} is recognized as the character specified by hexadeci-
mal code point. There may be any number of hexadecimal digits, but un-
- like other places that also use curly brackets, spaces are not allowed
- and would result in the string being interpreted as a literal. This
+ like other places that also use curly brackets, spaces are not allowed
+ and would result in the string being interpreted as a literal. This
syntax is from ECMAScript 6.
- The \N{U+hhh..} escape sequence is recognized only when PCRE2 is oper-
- ating in UTF mode. Perl also uses \N{name} to specify characters by
- Unicode name; PCRE2 does not support this. Note that when \N is not
+ The \N{U+hhh..} escape sequence is recognized only when PCRE2 is oper-
+ ating in UTF mode. Perl also uses \N{name} to specify characters by
+ Unicode name; PCRE2 does not support this. Note that when \N is not
followed by an opening brace (curly bracket) it has an entirely differ-
ent meaning, matching any character that is not a newline.
- There are some legacy applications where the escape sequence \r is ex-
- pected to match a newline. If the PCRE2_EXTRA_ESCAPED_CR_IS_LF option
- is set, \r in a pattern is converted to \n so that it matches a LF
+ There are some legacy applications where the escape sequence \r is ex-
+ pected to match a newline. If the PCRE2_EXTRA_ESCAPED_CR_IS_LF option
+ is set, \r in a pattern is converted to \n so that it matches a LF
(linefeed) instead of a CR (carriage return) character.
- An error occurs if \c is not followed by a character whose ASCII code
- point is in the range 32 to 126. The precise effect of \cx is as fol-
- lows: if x is a lower case letter, it is converted to upper case. Then
+ An error occurs if \c is not followed by a character whose ASCII code
+ point is in the range 32 to 126. The precise effect of \cx is as fol-
+ lows: if x is a lower case letter, it is converted to upper case. Then
bit 6 of the character (hex 40) is inverted. Thus \cA to \cZ become hex
- 01 to hex 1A (A is 41, Z is 5A), but \c{ becomes hex 3B ({ is 7B), and
- \c; becomes hex 7B (; is 3B). If the code unit following \c has a code
+ 01 to hex 1A (A is 41, Z is 5A), but \c{ becomes hex 3B ({ is 7B), and
+ \c; becomes hex 7B (; is 3B). If the code unit following \c has a code
point less than 32 or greater than 126, a compile-time error occurs.
- For differences in the way some escapes behave in EBCDIC environments,
+ For differences in the way some escapes behave in EBCDIC environments,
see section "EBCDIC environments" below.
Octal escapes and back references
- The escape \o must be followed by a sequence of octal digits, enclosed
- in braces. An error occurs if this is not the case. This escape pro-
- vides a way of specifying character code points as octal numbers
- greater than 0777, and it also allows octal numbers and backreferences
+ The escape \o must be followed by a sequence of octal digits, enclosed
+ in braces. An error occurs if this is not the case. This escape pro-
+ vides a way of specifying character code points as octal numbers
+ greater than 0777, and it also allows octal numbers and backreferences
to be unambiguously distinguished.
- If braces are not used, after \0 up to two further octal digits are
- read. However, if the PCRE2_EXTRA_NO_BS0 option is set, at least one
- more octal digit must follow \0 (use \00 to generate a NUL character).
- Make sure you supply two digits after the initial zero if the pattern
+ If braces are not used, after \0 up to two further octal digits are
+ read. However, if the PCRE2_EXTRA_NO_BS0 option is set, at least one
+ more octal digit must follow \0 (use \00 to generate a NUL character).
+ Make sure you supply two digits after the initial zero if the pattern
character that follows is itself an octal digit.
- Inside a character class, when a backslash is followed by any octal
- digit, up to three octal digits are read to generate a code point. Any
- subsequent digits stand for themselves. The sequences \8 and \9 are
+ Inside a character class, when a backslash is followed by any octal
+ digit, up to three octal digits are read to generate a code point. Any
+ subsequent digits stand for themselves. The sequences \8 and \9 are
treated as the literal characters "8" and "9".
Outside a character class, Perl's handling of a backslash followed by a
- digit other than 0 is complicated by ambiguity, and Perl has changed
+ digit other than 0 is complicated by ambiguity, and Perl has changed
over time, causing PCRE2 also to change. From PCRE2 release 10.45 there
- is an option called PCRE2_EXTRA_PYTHON_OCTAL that causes PCRE2 to use
- Python's unambiguous rules. The next two subsections describe the two
+ is an option called PCRE2_EXTRA_PYTHON_OCTAL that causes PCRE2 to use
+ Python's unambiguous rules. The next two subsections describe the two
sets of rules.
For greater clarity and unambiguity, it is best to avoid following \ by
- a digit greater than zero. Instead, use \o{...} or \x{...} to specify
+ a digit greater than zero. Instead, use \o{...} or \x{...} to specify
numerical character code points, and \g{...} to specify backreferences.
Perl rules for non-class backslash 1-9
- All the digits that follow the backslash are read as a decimal number.
- If the number is less than 10, begins with the digit 8 or 9, or if
+ All the digits that follow the backslash are read as a decimal number.
+ If the number is less than 10, begins with the digit 8 or 9, or if
there are at least that many previous capture groups in the expression,
- the entire sequence is taken as a back reference. Otherwise, up to
+ the entire sequence is taken as a back reference. Otherwise, up to
three octal digits are read to form a character code. For example:
\040 is another way of writing an ASCII space
@@ -7258,24 +7411,24 @@ BACKSLASH
the value 255 (decimal)
\81 is always a backreference
- Note that octal values of 100 or greater that are specified using this
- syntax must not be introduced by a leading zero, because no more than
+ Note that octal values of 100 or greater that are specified using this
+ syntax must not be introduced by a leading zero, because no more than
three octal digits are ever read.
Python rules for non_class backslash 1-9
- If there are at least three octal digits after the backslash, exactly
- three are read as an octal code point number, but the value must be no
- greater than \377, even in modes where higher code point values are
- supported. Any subsequent digits stand for themselves. If there are
- fewer than three octal digits, the sequence is taken as a decimal back
- reference. Thus, for example, \12 is always a back reference, indepen-
- dent of how many captures there are in the pattern. An error is gener-
+ If there are at least three octal digits after the backslash, exactly
+ three are read as an octal code point number, but the value must be no
+ greater than \377, even in modes where higher code point values are
+ supported. Any subsequent digits stand for themselves. If there are
+ fewer than three octal digits, the sequence is taken as a decimal back
+ reference. Thus, for example, \12 is always a back reference, indepen-
+ dent of how many captures there are in the pattern. An error is gener-
ated for a reference to a non-existent capturing group.
Constraints on character values
- Characters that are specified using octal or hexadecimal numbers are
+ Characters that are specified using octal or hexadecimal numbers are
limited to certain values, as follows:
8-bit non-UTF mode no greater than 0xff
@@ -7284,45 +7437,45 @@ BACKSLASH
All UTF modes no greater than 0x10ffff and a valid code point
Invalid Unicode code points are all those in the range 0xd800 to 0xdfff
- (the so-called "surrogate" code points). The check for these can be
- disabled by the caller of pcre2_compile() by setting the option
- PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES. However, this is possible only in
- UTF-8 and UTF-32 modes, because these values are not representable in
+ (the so-called "surrogate" code points). The check for these can be
+ disabled by the caller of pcre2_compile() by setting the option
+ PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES. However, this is possible only in
+ UTF-8 and UTF-32 modes, because these values are not representable in
UTF-16.
Escape sequences in character classes
All the sequences that define a single character value can be used both
- inside and outside character classes. In addition, inside a character
+ inside and outside character classes. In addition, inside a character
class, \b is interpreted as the backspace character (hex 08).
When not followed by an opening brace, \N is not allowed in a character
- class. \B, \R, and \X are not special inside a character class. Like
- other unrecognized alphabetic escape sequences, they cause an error.
+ class. \B, \R, and \X are not special inside a character class. Like
+ other unrecognized alphabetic escape sequences, they cause an error.
Outside a character class, these sequences have different meanings.
Unsupported escape sequences
- In Perl, the sequences \F, \l, \L, \u, and \U are recognized by its
- string handler and used to modify the case of following characters. By
- default, PCRE2 does not support these escape sequences in patterns.
- However, if either of the PCRE2_ALT_BSUX or PCRE2_EXTRA_ALT_BSUX op-
- tions is set, \U matches a "U" character, and \u can be used to define
+ In Perl, the sequences \F, \l, \L, \u, and \U are recognized by its
+ string handler and used to modify the case of following characters. By
+ default, PCRE2 does not support these escape sequences in patterns.
+ However, if either of the PCRE2_ALT_BSUX or PCRE2_EXTRA_ALT_BSUX op-
+ tions is set, \U matches a "U" character, and \u can be used to define
a character by code point, as described above.
Absolute and relative backreferences
The sequence \g followed by a signed or unsigned number, optionally en-
- closed in braces, is an absolute or relative backreference. A named
- backreference can be coded as \g{name}. Backreferences are discussed
+ closed in braces, is an absolute or relative backreference. A named
+ backreference can be coded as \g{name}. Backreferences are discussed
later, following the discussion of parenthesized groups.
Absolute and relative subroutine calls
- For compatibility with Oniguruma, the non-Perl syntax \g followed by a
+ For compatibility with Oniguruma, the non-Perl syntax \g followed by a
name or a number enclosed either in angle brackets or single quotes, is
- an alternative syntax for referencing a capture group as a subroutine.
- Details are discussed later. Note that \g{...} (Perl syntax) and
+ an alternative syntax for referencing a capture group as a subroutine.
+ Details are discussed later. Note that \g{...} (Perl syntax) and
\g<...> (Oniguruma syntax) are not synonymous. The former is a backref-
erence; the latter is a subroutine call.
@@ -7342,43 +7495,43 @@ BACKSLASH
\w any "word" character
\W any "non-word" character
- The \N escape sequence has the same meaning as the "." metacharacter
- when PCRE2_DOTALL is not set, but setting PCRE2_DOTALL does not change
+ The \N escape sequence has the same meaning as the "." metacharacter
+ when PCRE2_DOTALL is not set, but setting PCRE2_DOTALL does not change
the meaning of \N. Note that when \N is followed by an opening brace it
has a different meaning. See the section entitled "Non-printing charac-
- ters" above for details. Perl also uses \N{name} to specify characters
+ ters" above for details. Perl also uses \N{name} to specify characters
by Unicode name; PCRE2 does not support this.
- Each pair of lower and upper case escape sequences partitions the com-
- plete set of characters into two disjoint sets. Any given character
- matches one, and only one, of each pair. The sequences can appear both
- inside and outside character classes. They each match one character of
- the appropriate type. If the current matching point is at the end of
- the subject string, all of them fail, because there is no character to
+ Each pair of lower and upper case escape sequences partitions the com-
+ plete set of characters into two disjoint sets. Any given character
+ matches one, and only one, of each pair. The sequences can appear both
+ inside and outside character classes. They each match one character of
+ the appropriate type. If the current matching point is at the end of
+ the subject string, all of them fail, because there is no character to
match.
- The default \s characters are HT (9), LF (10), VT (11), FF (12), CR
- (13), and space (32), which are defined as white space in the "C" lo-
- cale. This list may vary if locale-specific matching is taking place.
- For example, in some locales the "non-breaking space" character (\xA0)
+ The default \s characters are HT (9), LF (10), VT (11), FF (12), CR
+ (13), and space (32), which are defined as white space in the "C" lo-
+ cale. This list may vary if locale-specific matching is taking place.
+ For example, in some locales the "non-breaking space" character (\xA0)
is recognized as white space, and in others the VT character is not.
- A "word" character is an underscore or any character that is a letter
- or digit. By default, the definition of letters and digits is con-
+ A "word" character is an underscore or any character that is a letter
+ or digit. By default, the definition of letters and digits is con-
trolled by PCRE2's low-valued character tables, and may vary if locale-
specific matching is taking place (see "Locale support" in the pcre2api
- page). For example, in a French locale such as "fr_FR" in Unix-like
- systems, or "french" in Windows, some character codes greater than 127
- are used for accented letters, and these are then matched by \w. The
+ page). For example, in a French locale such as "fr_FR" in Unix-like
+ systems, or "french" in Windows, some character codes greater than 127
+ are used for accented letters, and these are then matched by \w. The
use of locales with Unicode is discouraged.
- By default, characters whose code points are greater than 127 never
+ By default, characters whose code points are greater than 127 never
match \d, \s, or \w, and always match \D, \S, and \W, although this may
- be different for characters in the range 128-255 when locale-specific
- matching is happening. These escape sequences retain their original
- meanings from before Unicode support was available, mainly for effi-
- ciency reasons. If the PCRE2_UCP option is set, the behaviour is
- changed so that Unicode properties are used to determine character
+ be different for characters in the range 128-255 when locale-specific
+ matching is happening. These escape sequences retain their original
+ meanings from before Unicode support was available, mainly for effi-
+ ciency reasons. If the PCRE2_UCP option is set, the behaviour is
+ changed so that Unicode properties are used to determine character
types, as follows:
\d any character that matches \p{Nd} (decimal digit)
@@ -7386,25 +7539,25 @@ BACKSLASH
\w any character that matches \p{L}, \p{N}, \p{Mn}, or \p{Pc}
The addition of \p{Mn} (non-spacing mark) and the replacement of an ex-
- plicit test for underscore with a test for \p{Pc} (connector punctua-
+ plicit test for underscore with a test for \p{Pc} (connector punctua-
tion) happened in PCRE2 release 10.43. This brings PCRE2 into line with
Perl.
- The upper case escapes match the inverse sets of characters. Note that
- \d matches only decimal digits, whereas \w matches any Unicode digit,
+ The upper case escapes match the inverse sets of characters. Note that
+ \d matches only decimal digits, whereas \w matches any Unicode digit,
as well as other character categories. Note also that PCRE2_UCP affects
- \b, and \B because they are defined in terms of \w and \W. Matching
+ \b, and \B because they are defined in terms of \w and \W. Matching
these sequences is noticeably slower when PCRE2_UCP is set.
- The effect of PCRE2_UCP on any one of these escape sequences can be
- negated by the options PCRE2_EXTRA_ASCII_BSD, PCRE2_EXTRA_ASCII_BSS,
- and PCRE2_EXTRA_ASCII_BSW, respectively. These options can be set and
- reset within a pattern by means of an internal option setting (see be-
+ The effect of PCRE2_UCP on any one of these escape sequences can be
+ negated by the options PCRE2_EXTRA_ASCII_BSD, PCRE2_EXTRA_ASCII_BSS,
+ and PCRE2_EXTRA_ASCII_BSW, respectively. These options can be set and
+ reset within a pattern by means of an internal option setting (see be-
low).
- The sequences \h, \H, \v, and \V, in contrast to the other sequences,
- which match only ASCII characters by default, always match a specific
- list of code points, whether or not PCRE2_UCP is set. The horizontal
+ The sequences \h, \H, \v, and \V, in contrast to the other sequences,
+ which match only ASCII characters by default, always match a specific
+ list of code points, whether or not PCRE2_UCP is set. The horizontal
space characters are:
U+0009 Horizontal tab (HT)
@@ -7437,36 +7590,36 @@ BACKSLASH
U+2028 Line separator
U+2029 Paragraph separator
- In 8-bit, non-UTF-8 mode, only the characters with code points less
+ In 8-bit, non-UTF-8 mode, only the characters with code points less
than 256 are relevant.
Newline sequences
- Outside a character class, by default, the escape sequence \R matches
- any Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent
+ Outside a character class, by default, the escape sequence \R matches
+ any Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent
to the following:
(?>\r\n|\n|\x0b|\f|\r|\x85)
This is an example of an "atomic group", details of which are given be-
- low. This particular group matches either the two-character sequence
- CR followed by LF, or one of the single characters LF (linefeed,
- U+000A), VT (vertical tab, U+000B), FF (form feed, U+000C), CR (car-
- riage return, U+000D), or NEL (next line, U+0085). Because this is an
- atomic group, the two-character sequence is treated as a single unit
+ low. This particular group matches either the two-character sequence
+ CR followed by LF, or one of the single characters LF (linefeed,
+ U+000A), VT (vertical tab, U+000B), FF (form feed, U+000C), CR (car-
+ riage return, U+000D), or NEL (next line, U+0085). Because this is an
+ atomic group, the two-character sequence is treated as a single unit
that cannot be split.
In other modes, two additional characters whose code points are greater
than 255 are added: LS (line separator, U+2028) and PS (paragraph sepa-
- rator, U+2029). Unicode support is not needed for these characters to
+ rator, U+2029). Unicode support is not needed for these characters to
be recognized.
It is possible to restrict \R to match only CR, LF, or CRLF (instead of
- the complete set of Unicode line endings) by setting the option
- PCRE2_BSR_ANYCRLF at compile time. (BSR is an abbreviation for "back-
+ the complete set of Unicode line endings) by setting the option
+ PCRE2_BSR_ANYCRLF at compile time. (BSR is an abbreviation for "back-
slash R".) This can be made the default when PCRE2 is built; if this is
- the case, the other behaviour can be requested via the PCRE2_BSR_UNI-
- CODE option. It is also possible to specify these settings by starting
+ the case, the other behaviour can be requested via the PCRE2_BSR_UNI-
+ CODE option. It is also possible to specify these settings by starting
a pattern string with one of the following sequences:
(*BSR_ANYCRLF) CR, LF, or CRLF only
@@ -7474,33 +7627,33 @@ BACKSLASH
These override the default and the options given to the compiling func-
tion. Note that these special settings, which are not Perl-compatible,
- are recognized only at the very start of a pattern, and that they must
- be in upper case. If more than one of them is present, the last one is
+ are recognized only at the very start of a pattern, and that they must
+ be in upper case. If more than one of them is present, the last one is
used. They can be combined with a change of newline convention; for ex-
ample, a pattern can start with:
(*ANY)(*BSR_ANYCRLF)
- They can also be combined with the (*UTF) or (*UCP) special sequences.
- Inside a character class, \R is treated as an unrecognized escape se-
+ They can also be combined with the (*UTF) or (*UCP) special sequences.
+ Inside a character class, \R is treated as an unrecognized escape se-
quence, and causes an error.
Unicode character properties
- When PCRE2 is built with Unicode support (the default), three addi-
- tional escape sequences that match characters with specific properties
+ When PCRE2 is built with Unicode support (the default), three addi-
+ tional escape sequences that match characters with specific properties
are available. They can be used in any mode, though in 8-bit and 16-bit
- non-UTF modes these sequences are of course limited to testing charac-
- ters whose code points are less than U+0100 or U+10000, respectively.
- In 32-bit non-UTF mode, code points greater than 0x10ffff (the Unicode
- limit) may be encountered. These are all treated as being in the Un-
+ non-UTF modes these sequences are of course limited to testing charac-
+ ters whose code points are less than U+0100 or U+10000, respectively.
+ In 32-bit non-UTF mode, code points greater than 0x10ffff (the Unicode
+ limit) may be encountered. These are all treated as being in the Un-
known script and with an unassigned type.
- Matching characters by Unicode property is not fast, because PCRE2 has
- to do a multistage table lookup in order to find a character's prop-
+ Matching characters by Unicode property is not fast, because PCRE2 has
+ to do a multistage table lookup in order to find a character's prop-
erty. That is why the traditional escape sequences such as \d and \w do
- not use Unicode properties in PCRE2 by default, though you can make
- them do so by setting the PCRE2_UCP option or by starting the pattern
+ not use Unicode properties in PCRE2 by default, though you can make
+ them do so by setting the PCRE2_UCP option or by starting the pattern
with (*UCP).
The extra escape sequences that provide property support are:
@@ -7509,20 +7662,20 @@ BACKSLASH
\P{xx} a character without the xx property
\X a Unicode extended grapheme cluster
- For compatibility with Perl, negation can be specified by including a
- circumflex between the opening brace and the property. For example,
+ For compatibility with Perl, negation can be specified by including a
+ circumflex between the opening brace and the property. For example,
\p{^Lu} is the same as \P{Lu}.
- In accordance with Unicode's "loose matching" rules, ASCII white space
+ In accordance with Unicode's "loose matching" rules, ASCII white space
characters, hyphens, and underscores are ignored in the properties rep-
resented by xx above. As well as the space character, ASCII white space
can be tab, linefeed, vertical tab, formfeed, or carriage return.
- Some properties are specified as a name only; others as a name and a
- value, separated by a colon or an equals sign. The names and values
- consist of ASCII letters and digits (with one Perl-specific exception,
- see below). They are not case sensitive. Note, however, that the es-
- capes themselves, \p and \P, are case sensitive. There are abbrevia-
+ Some properties are specified as a name only; others as a name and a
+ value, separated by a colon or an equals sign. The names and values
+ consist of ASCII letters and digits (with one Perl-specific exception,
+ see below). They are not case sensitive. Note, however, that the es-
+ capes themselves, \p and \P, are case sensitive. There are abbrevia-
tions for many names. The following examples are all equivalent:
\p{bidiclass=al}
@@ -7531,30 +7684,30 @@ BACKSLASH
\p{ Bi-di class = Al }
\P{ ^ Bi-di class = Al }
- There is support for Unicode script names, Unicode general category
- properties, "Any", which matches any character (including newline),
- Bidi_Class, a number of binary (yes/no) properties, and some special
+ There is support for Unicode script names, Unicode general category
+ properties, "Any", which matches any character (including newline),
+ Bidi_Class, a number of binary (yes/no) properties, and some special
PCRE2 properties (described below). Certain other Perl properties such
- as "InMusicalSymbols" are not supported by PCRE2. Note that \P{Any}
+ as "InMusicalSymbols" are not supported by PCRE2. Note that \P{Any}
does not match any characters, so always causes a match failure.
Script properties for \p and \P
There are three different syntax forms for matching a script. Each Uni-
- code character has a basic script and, optionally, a list of other
+ code character has a basic script and, optionally, a list of other
scripts ("Script Extensions") with which it is commonly used. Using the
Adlam script as an example, \p{sc:Adlam} matches characters whose basic
script is Adlam, whereas \p{scx:Adlam} matches, in addition, characters
- that have Adlam in their extensions list. The full names "script" and
- "script extensions" for the property types are recognized and, as for
- all property specifications, an equals sign is an alternative to the
- colon. If a script name is given without a property type, for example,
- \p{Adlam}, it is treated as \p{scx:Adlam}. Perl changed to this inter-
+ that have Adlam in their extensions list. The full names "script" and
+ "script extensions" for the property types are recognized and, as for
+ all property specifications, an equals sign is an alternative to the
+ colon. If a script name is given without a property type, for example,
+ \p{Adlam}, it is treated as \p{scx:Adlam}. Perl changed to this inter-
pretation at release 5.26 and PCRE2 changed at release 10.40.
Unassigned characters (and in non-UTF 32-bit mode, characters with code
points greater than 0x10FFFF) are assigned the "Unknown" script. Others
- that are not part of an identified script are lumped together as "Com-
+ that are not part of an identified script are lumped together as "Com-
mon". The current list of recognized script names and their 4-character
abbreviations can be obtained by running this command:
@@ -7564,10 +7717,10 @@ BACKSLASH
The general category property for \p and \P
Each character has exactly one Unicode general category property, spec-
- ified by a two-letter abbreviation. If only one letter is specified
- with \p or \P, it includes all the general category properties that
- start with that letter. In this case, in the absence of negation, the
- curly brackets in the escape sequence are optional; these two examples
+ ified by a two-letter abbreviation. If only one letter is specified
+ with \p or \P, it includes all the general category properties that
+ start with that letter. In this case, in the absence of negation, the
+ curly brackets in the escape sequence are optional; these two examples
have the same effect:
\p{L}
@@ -7620,25 +7773,25 @@ BACKSLASH
Zp Paragraph separator
Zs Space separator
- Perl originally used the name L& for the Lc property. This is still
- supported by Perl, but discouraged. PCRE2 also still supports it. This
- property matches any character that has the Lu, Ll, or Lt property, in
- other words, any letter that is not classified as a modifier or
- "other". From release 10.45 of PCRE2 the properties Lu, Ll, and Lt are
- all treated as Lc when case-independent matching is set by the
- PCRE2_CASELESS option or (?i) within the pattern. The other properties
+ Perl originally used the name L& for the Lc property. This is still
+ supported by Perl, but discouraged. PCRE2 also still supports it. This
+ property matches any character that has the Lu, Ll, or Lt property, in
+ other words, any letter that is not classified as a modifier or
+ "other". From release 10.45 of PCRE2 the properties Lu, Ll, and Lt are
+ all treated as Lc when case-independent matching is set by the
+ PCRE2_CASELESS option or (?i) within the pattern. The other properties
are not affected by caseless matching.
- The Cs (Surrogate) property applies only to characters whose code
- points are in the range U+D800 to U+DFFF. These characters are no dif-
- ferent to any other character when PCRE2 is not in UTF mode (using the
- 16-bit or 32-bit library). However, they are not valid in Unicode
+ The Cs (Surrogate) property applies only to characters whose code
+ points are in the range U+D800 to U+DFFF. These characters are no dif-
+ ferent to any other character when PCRE2 is not in UTF mode (using the
+ 16-bit or 32-bit library). However, they are not valid in Unicode
strings and so cannot be tested by PCRE2 in UTF mode, unless UTF valid-
- ity checking has been turned off (see the discussion of
+ ity checking has been turned off (see the discussion of
PCRE2_NO_UTF_CHECK in the pcre2api page).
- The long synonyms for property names that Perl supports (such as
- \p{Letter}) are not supported by PCRE2, nor is it permitted to prefix
+ The long synonyms for property names that Perl supports (such as
+ \p{Letter}) are not supported by PCRE2, nor is it permitted to prefix
any of these properties with "Is".
No character that is in the Unicode table has the Cn (unassigned) prop-
@@ -7647,9 +7800,9 @@ BACKSLASH
Binary (yes/no) properties for \p and \P
- Unicode defines a number of binary properties, that is, properties
- whose only values are true or false. You can obtain a list of those
- that are recognized by \p and \P, along with their abbreviations, by
+ Unicode defines a number of binary properties, that is, properties
+ whose only values are true or false. You can obtain a list of those
+ that are recognized by \p and \P, along with their abbreviations, by
running this command:
pcre2test -LP
@@ -7686,63 +7839,63 @@ BACKSLASH
S segment separator
WS white space
- As in all property specifications, an equals sign may be used instead
- of a colon and the class names are case-insensitive. Only the short
- names listed above are recognized; PCRE2 does not at present support
+ As in all property specifications, an equals sign may be used instead
+ of a colon and the class names are case-insensitive. Only the short
+ names listed above are recognized; PCRE2 does not at present support
any long alternatives.
Extended grapheme clusters
- The \X escape matches any number of Unicode characters that form an
+ The \X escape matches any number of Unicode characters that form an
"extended grapheme cluster", and treats the sequence as an atomic group
- (see below). Unicode supports various kinds of composite character by
- giving each character a grapheme breaking property, and having rules
+ (see below). Unicode supports various kinds of composite character by
+ giving each character a grapheme breaking property, and having rules
that use these properties to define the boundaries of extended grapheme
- clusters. The rules are defined in Unicode Standard Annex 29, "Unicode
- Text Segmentation". Unicode 11.0.0 abandoned the use of some previous
- properties that had been used for emojis. Instead it introduced vari-
- ous emoji-specific properties. PCRE2 uses only the Extended Picto-
+ clusters. The rules are defined in Unicode Standard Annex 29, "Unicode
+ Text Segmentation". Unicode 11.0.0 abandoned the use of some previous
+ properties that had been used for emojis. Instead it introduced vari-
+ ous emoji-specific properties. PCRE2 uses only the Extended Picto-
graphic property.
- \X always matches at least one character. Then it decides whether to
+ \X always matches at least one character. Then it decides whether to
add additional characters according to the following rules for ending a
cluster:
1. End at the end of the subject string.
- 2. Do not end between CR and LF; otherwise end after any control char-
+ 2. Do not end between CR and LF; otherwise end after any control char-
acter.
- 3. Do not break Hangul (a Korean script) syllable sequences. Hangul
- characters are of five types: L, V, T, LV, and LVT. An L character may
- be followed by an L, V, LV, or LVT character; an LV or V character may
- be followed by a V or T character; an LVT or T character may be fol-
+ 3. Do not break Hangul (a Korean script) syllable sequences. Hangul
+ characters are of five types: L, V, T, LV, and LVT. An L character may
+ be followed by an L, V, LV, or LVT character; an LV or V character may
+ be followed by a V or T character; an LVT or T character may be fol-
lowed only by a T character.
4. Do not end before extending characters or spacing marks or the zero-
- width joiner (ZWJ) character. Characters with the "mark" property al-
+ width joiner (ZWJ) character. Characters with the "mark" property al-
ways have the "extend" grapheme breaking property.
5. Do not end after prepend characters.
- 6. Do not end within emoji modifier sequences or emoji ZWJ (zero-width
- joiner) sequences. An emoji ZWJ sequence consists of a character with
- the Extended_Pictographic property, optionally followed by one or more
- characters with the Extend property, followed by the ZWJ character,
+ 6. Do not end within emoji modifier sequences or emoji ZWJ (zero-width
+ joiner) sequences. An emoji ZWJ sequence consists of a character with
+ the Extended_Pictographic property, optionally followed by one or more
+ characters with the Extend property, followed by the ZWJ character,
followed by another Extended_Pictographic character.
- 7. Do not break within emoji flag sequences. That is, do not break be-
- tween regional indicator (RI) characters if there are an odd number of
+ 7. Do not break within emoji flag sequences. That is, do not break be-
+ tween regional indicator (RI) characters if there are an odd number of
RI characters before the break point.
8. Otherwise, end the cluster.
PCRE2's additional properties
- As well as the standard Unicode properties described above, PCRE2 sup-
+ As well as the standard Unicode properties described above, PCRE2 sup-
ports four more that make it possible to convert traditional escape se-
- quences such as \w and \s to use Unicode properties. PCRE2 uses these
- non-standard, non-Perl properties internally when PCRE2_UCP is set.
+ quences such as \w and \s to use Unicode properties. PCRE2 uses these
+ non-standard, non-Perl properties internally when PCRE2_UCP is set.
However, they may also be used explicitly. These properties are:
Xan Any alphanumeric character
@@ -7750,74 +7903,74 @@ BACKSLASH
Xsp Any Perl space character
Xwd Any Perl "word" character
- Xan matches characters that have either the L (letter) or the N (num-
- ber) property. Xps matches the characters tab, linefeed, vertical tab,
- form feed, or carriage return, and any other character that has the Z
- (separator) property (this includes the space character). Xsp is the
+ Xan matches characters that have either the L (letter) or the N (num-
+ ber) property. Xps matches the characters tab, linefeed, vertical tab,
+ form feed, or carriage return, and any other character that has the Z
+ (separator) property (this includes the space character). Xsp is the
same as Xps; in PCRE1 it used to exclude vertical tab, for Perl compat-
ibility, but Perl changed. Xwd matches the same characters as Xan, plus
- those that match Mn (non-spacing mark) or Pc (connector punctuation,
+ those that match Mn (non-spacing mark) or Pc (connector punctuation,
which includes underscore).
- There is another non-standard property, Xuc, which matches any charac-
- ter that can be represented by a Universal Character Name in C++ and
- other programming languages. These are the characters $, @, ` (grave
- accent), and all characters with Unicode code points greater than or
- equal to U+00A0, except for the surrogates U+D800 to U+DFFF. Note that
- most base (ASCII) characters are excluded. (Universal Character Names
- are of the form \uHHHH or \UHHHHHHHH where H is a hexadecimal digit.
+ There is another non-standard property, Xuc, which matches any charac-
+ ter that can be represented by a Universal Character Name in C++ and
+ other programming languages. These are the characters $, @, ` (grave
+ accent), and all characters with Unicode code points greater than or
+ equal to U+00A0, except for the surrogates U+D800 to U+DFFF. Note that
+ most base (ASCII) characters are excluded. (Universal Character Names
+ are of the form \uHHHH or \UHHHHHHHH where H is a hexadecimal digit.
Note that the Xuc property does not match these sequences but the char-
acters that they represent.)
Resetting the match start
- In normal use, the escape sequence \K causes any previously matched
+ In normal use, the escape sequence \K causes any previously matched
characters not to be included in the final matched sequence that is re-
turned. For example, the pattern:
foo\Kbar
- matches "foobar", but reports that it has matched "bar". \K does not
+ matches "foobar", but reports that it has matched "bar". \K does not
interact with anchoring in any way. The pattern:
^foo\Kbar
- matches only when the subject begins with "foobar" (in single line
- mode), though it again reports the matched string as "bar". This fea-
- ture is similar to a lookbehind assertion (described below), but the
+ matches only when the subject begins with "foobar" (in single line
+ mode), though it again reports the matched string as "bar". This fea-
+ ture is similar to a lookbehind assertion (described below), but the
part of the pattern that precedes \K is not constrained to match a lim-
- ited number of characters, as is required for a lookbehind assertion.
- The use of \K does not interfere with the setting of captured sub-
+ ited number of characters, as is required for a lookbehind assertion.
+ The use of \K does not interfere with the setting of captured sub-
strings. For example, when the pattern
(foo)\Kbar
matches "foobar", the first substring is still set to "foo".
- From version 5.32.0 Perl forbids the use of \K in lookaround asser-
- tions. From release 10.38 PCRE2 also forbids this by default. However,
- the PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK option can be used when calling
- pcre2_compile() to re-enable the previous behaviour. When this option
+ From version 5.32.0 Perl forbids the use of \K in lookaround asser-
+ tions. From release 10.38 PCRE2 also forbids this by default. However,
+ the PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK option can be used when calling
+ pcre2_compile() to re-enable the previous behaviour. When this option
is set, \K is acted upon when it occurs inside positive assertions, but
- is ignored in negative assertions. Note that when a pattern such as
- (?=ab\K) matches, the reported start of the match can be greater than
- the end of the match. Using \K in a lookbehind assertion at the start
- of a pattern can also lead to odd effects. For example, consider this
+ is ignored in negative assertions. Note that when a pattern such as
+ (?=ab\K) matches, the reported start of the match can be greater than
+ the end of the match. Using \K in a lookbehind assertion at the start
+ of a pattern can also lead to odd effects. For example, consider this
pattern:
(?<=\Kfoo)bar
- If the subject is "foobar", a call to pcre2_match() with a starting
- offset of 3 succeeds and reports the matching string as "foobar", that
- is, the start of the reported match is earlier than where the match
+ If the subject is "foobar", a call to pcre2_match() with a starting
+ offset of 3 succeeds and reports the matching string as "foobar", that
+ is, the start of the reported match is earlier than where the match
started.
Simple assertions
- The final use of backslash is for certain simple assertions. An asser-
- tion specifies a condition that has to be met at a particular point in
- a match, without consuming any characters from the subject string. The
- use of groups for more complicated assertions is described below. The
+ The final use of backslash is for certain simple assertions. An asser-
+ tion specifies a condition that has to be met at a particular point in
+ a match, without consuming any characters from the subject string. The
+ use of groups for more complicated assertions is described below. The
backslashed assertions are:
\b matches at a word boundary
@@ -7828,193 +7981,193 @@ BACKSLASH
\z matches only at the end of the subject
\G matches at the first matching position in the subject
- Inside a character class, \b has a different meaning; it matches the
- backspace character. If any other of these assertions appears in a
+ Inside a character class, \b has a different meaning; it matches the
+ backspace character. If any other of these assertions appears in a
character class, an "invalid escape sequence" error is generated.
- A word boundary is a position in the subject string where the current
- character and the previous character do not both match \w or \W (i.e.
- one matches \w and the other matches \W), or the start or end of the
- string if the first or last character matches \w, respectively. When
- PCRE2 is built with Unicode support, the meanings of \w and \W can be
+ A word boundary is a position in the subject string where the current
+ character and the previous character do not both match \w or \W (i.e.
+ one matches \w and the other matches \W), or the start or end of the
+ string if the first or last character matches \w, respectively. When
+ PCRE2 is built with Unicode support, the meanings of \w and \W can be
changed by setting the PCRE2_UCP option. When this is done, it also af-
- fects \b and \B. Neither PCRE2 nor Perl has a separate "start of word"
- or "end of word" metasequence. However, whatever follows \b normally
- determines which it is. For example, the fragment \ba matches "a" at
+ fects \b and \B. Neither PCRE2 nor Perl has a separate "start of word"
+ or "end of word" metasequence. However, whatever follows \b normally
+ determines which it is. For example, the fragment \ba matches "a" at
the start of a word.
- The \A, \Z, and \z assertions differ from the traditional circumflex
+ The \A, \Z, and \z assertions differ from the traditional circumflex
and dollar (described in the next section) in that they only ever match
- at the very start and end of the subject string, whatever options are
- set. Thus, they are independent of multiline mode. These three asser-
- tions are not affected by the PCRE2_NOTBOL or PCRE2_NOTEOL options,
- which affect only the behaviour of the circumflex and dollar metachar-
- acters. However, if the startoffset argument of pcre2_match() is non-
- zero, indicating that matching is to start at a point other than the
- beginning of the subject, \A can never match. The difference between
- \Z and \z is that \Z matches before a newline at the end of the string
+ at the very start and end of the subject string, whatever options are
+ set. Thus, they are independent of multiline mode. These three asser-
+ tions are not affected by the PCRE2_NOTBOL or PCRE2_NOTEOL options,
+ which affect only the behaviour of the circumflex and dollar metachar-
+ acters. However, if the startoffset argument of pcre2_match() is non-
+ zero, indicating that matching is to start at a point other than the
+ beginning of the subject, \A can never match. The difference between
+ \Z and \z is that \Z matches before a newline at the end of the string
as well as at the very end, whereas \z matches only at the end.
- The \G assertion is true only when the current matching position is at
- the start point of the matching process, as specified by the startoff-
- set argument of pcre2_match(). It differs from \A when the value of
- startoffset is non-zero. By calling pcre2_match() multiple times with
- appropriate arguments, you can mimic Perl's /g option, and it is in
+ The \G assertion is true only when the current matching position is at
+ the start point of the matching process, as specified by the startoff-
+ set argument of pcre2_match(). It differs from \A when the value of
+ startoffset is non-zero. By calling pcre2_match() multiple times with
+ appropriate arguments, you can mimic Perl's /g option, and it is in
this kind of implementation where \G can be useful.
- Note, however, that PCRE2's implementation of \G, being true at the
- starting character of the matching process, is subtly different from
- Perl's, which defines it as true at the end of the previous match. In
- Perl, these can be different when the previously matched string was
+ Note, however, that PCRE2's implementation of \G, being true at the
+ starting character of the matching process, is subtly different from
+ Perl's, which defines it as true at the end of the previous match. In
+ Perl, these can be different when the previously matched string was
empty. Because PCRE2 does just one match at a time, it cannot reproduce
this behaviour.
- If all the alternatives of a pattern begin with \G, the expression is
+ If all the alternatives of a pattern begin with \G, the expression is
anchored to the starting match position, and the "anchored" flag is set
in the compiled regular expression.
CIRCUMFLEX AND DOLLAR
- The circumflex and dollar metacharacters are zero-width assertions.
- That is, they test for a particular condition being true without con-
+ The circumflex and dollar metacharacters are zero-width assertions.
+ That is, they test for a particular condition being true without con-
suming any characters from the subject string. These two metacharacters
- are concerned with matching the starts and ends of lines. If the new-
- line convention is set so that only the two-character sequence CRLF is
- recognized as a newline, isolated CR and LF characters are treated as
+ are concerned with matching the starts and ends of lines. If the new-
+ line convention is set so that only the two-character sequence CRLF is
+ recognized as a newline, isolated CR and LF characters are treated as
ordinary data characters, and are not recognized as newlines.
Outside a character class, in the default matching mode, the circumflex
- character is an assertion that is true only if the current matching
- point is at the start of the subject string. If the startoffset argu-
- ment of pcre2_match() is non-zero, or if PCRE2_NOTBOL is set, circum-
- flex can never match if the PCRE2_MULTILINE option is unset. Inside a
- character class, circumflex has an entirely different meaning (see be-
+ character is an assertion that is true only if the current matching
+ point is at the start of the subject string. If the startoffset argu-
+ ment of pcre2_match() is non-zero, or if PCRE2_NOTBOL is set, circum-
+ flex can never match if the PCRE2_MULTILINE option is unset. Inside a
+ character class, circumflex has an entirely different meaning (see be-
low).
- Circumflex need not be the first character of the pattern if a number
- of alternatives are involved, but it should be the first thing in each
- alternative in which it appears if the pattern is ever to match that
- branch. If all possible alternatives start with a circumflex, that is,
- if the pattern is constrained to match only at the start of the sub-
- ject, it is said to be an "anchored" pattern. (There are also other
+ Circumflex need not be the first character of the pattern if a number
+ of alternatives are involved, but it should be the first thing in each
+ alternative in which it appears if the pattern is ever to match that
+ branch. If all possible alternatives start with a circumflex, that is,
+ if the pattern is constrained to match only at the start of the sub-
+ ject, it is said to be an "anchored" pattern. (There are also other
constructs that can cause a pattern to be anchored.)
- The dollar character is an assertion that is true only if the current
- matching point is at the end of the subject string, or immediately be-
- fore a newline at the end of the string (by default), unless PCRE2_NO-
- TEOL is set. Note, however, that it does not actually match the new-
- line. Dollar need not be the last character of the pattern if a number
- of alternatives are involved, but it should be the last item in any
- branch in which it appears. Dollar has no special meaning in a charac-
+ The dollar character is an assertion that is true only if the current
+ matching point is at the end of the subject string, or immediately be-
+ fore a newline at the end of the string (by default), unless PCRE2_NO-
+ TEOL is set. Note, however, that it does not actually match the new-
+ line. Dollar need not be the last character of the pattern if a number
+ of alternatives are involved, but it should be the last item in any
+ branch in which it appears. Dollar has no special meaning in a charac-
ter class.
- The meaning of dollar can be changed so that it matches only at the
- very end of the string, by setting the PCRE2_DOLLAR_ENDONLY option at
+ The meaning of dollar can be changed so that it matches only at the
+ very end of the string, by setting the PCRE2_DOLLAR_ENDONLY option at
compile time. This does not affect the \Z assertion.
The meanings of the circumflex and dollar metacharacters are changed if
- the PCRE2_MULTILINE option is set. When this is the case, a dollar
- character matches before any newlines in the string, as well as at the
- very end, and a circumflex matches immediately after internal newlines
- as well as at the start of the subject string. It does not match after
- a newline that ends the string, for compatibility with Perl. However,
+ the PCRE2_MULTILINE option is set. When this is the case, a dollar
+ character matches before any newlines in the string, as well as at the
+ very end, and a circumflex matches immediately after internal newlines
+ as well as at the start of the subject string. It does not match after
+ a newline that ends the string, for compatibility with Perl. However,
this can be changed by setting the PCRE2_ALT_CIRCUMFLEX option.
- For example, the pattern /^abc$/ matches the subject string "def\nabc"
- (where \n represents a newline) in multiline mode, but not otherwise.
- Consequently, patterns that are anchored in single line mode because
- all branches start with ^ are not anchored in multiline mode, and a
- match for circumflex is possible when the startoffset argument of
- pcre2_match() is non-zero. The PCRE2_DOLLAR_ENDONLY option is ignored
+ For example, the pattern /^abc$/ matches the subject string "def\nabc"
+ (where \n represents a newline) in multiline mode, but not otherwise.
+ Consequently, patterns that are anchored in single line mode because
+ all branches start with ^ are not anchored in multiline mode, and a
+ match for circumflex is possible when the startoffset argument of
+ pcre2_match() is non-zero. The PCRE2_DOLLAR_ENDONLY option is ignored
if PCRE2_MULTILINE is set.
- When the newline convention (see "Newline conventions" below) recog-
- nizes the two-character sequence CRLF as a newline, this is preferred,
- even if the single characters CR and LF are also recognized as new-
- lines. For example, if the newline convention is "any", a multiline
- mode circumflex matches before "xyz" in the string "abc\r\nxyz" rather
- than after CR, even though CR on its own is a valid newline. (It also
+ When the newline convention (see "Newline conventions" below) recog-
+ nizes the two-character sequence CRLF as a newline, this is preferred,
+ even if the single characters CR and LF are also recognized as new-
+ lines. For example, if the newline convention is "any", a multiline
+ mode circumflex matches before "xyz" in the string "abc\r\nxyz" rather
+ than after CR, even though CR on its own is a valid newline. (It also
matches at the very start of the string, of course.)
- Note that the sequences \A, \Z, and \z can be used to match the start
- and end of the subject in both modes, and if all branches of a pattern
- start with \A it is always anchored, whether or not PCRE2_MULTILINE is
+ Note that the sequences \A, \Z, and \z can be used to match the start
+ and end of the subject in both modes, and if all branches of a pattern
+ start with \A it is always anchored, whether or not PCRE2_MULTILINE is
set.
FULL STOP (PERIOD, DOT) AND \N
Outside a character class, a dot in the pattern matches any one charac-
- ter in the subject string except (by default) a character that signi-
+ ter in the subject string except (by default) a character that signi-
fies the end of a line. One or more characters may be specified as line
terminators (see "Newline conventions" above).
- Dot never matches a single line-ending character. When the two-charac-
- ter sequence CRLF is the only line ending, dot does not match CR if it
- is immediately followed by LF, but otherwise it matches all characters
- (including isolated CRs and LFs). When ANYCRLF is selected for line
- endings, no occurrences of CR of LF match dot. When all Unicode line
+ Dot never matches a single line-ending character. When the two-charac-
+ ter sequence CRLF is the only line ending, dot does not match CR if it
+ is immediately followed by LF, but otherwise it matches all characters
+ (including isolated CRs and LFs). When ANYCRLF is selected for line
+ endings, no occurrences of CR of LF match dot. When all Unicode line
endings are being recognized, dot does not match CR or LF or any of the
other line ending characters.
- The behaviour of dot with regard to newlines can be changed. If the
- PCRE2_DOTALL option is set, a dot matches any one character, without
- exception. If the two-character sequence CRLF is present in the sub-
+ The behaviour of dot with regard to newlines can be changed. If the
+ PCRE2_DOTALL option is set, a dot matches any one character, without
+ exception. If the two-character sequence CRLF is present in the sub-
ject string, it takes two dots to match it.
- The handling of dot is entirely independent of the handling of circum-
- flex and dollar, the only relationship being that they both involve
+ The handling of dot is entirely independent of the handling of circum-
+ flex and dollar, the only relationship being that they both involve
newlines. Dot has no special meaning in a character class.
- The escape sequence \N when not followed by an opening brace behaves
- like a dot, except that it is not affected by the PCRE2_DOTALL option.
- In other words, it matches any character except one that signifies the
+ The escape sequence \N when not followed by an opening brace behaves
+ like a dot, except that it is not affected by the PCRE2_DOTALL option.
+ In other words, it matches any character except one that signifies the
end of a line.
When \N is followed by an opening brace it has a different meaning. See
- the section entitled "Non-printing characters" above for details. Perl
- also uses \N{name} to specify characters by Unicode name; PCRE2 does
+ the section entitled "Non-printing characters" above for details. Perl
+ also uses \N{name} to specify characters by Unicode name; PCRE2 does
not support this.
MATCHING A SINGLE CODE UNIT
- Outside a character class, the escape sequence \C matches any one code
- unit, whether or not a UTF mode is set. In the 8-bit library, one code
- unit is one byte; in the 16-bit library it is a 16-bit unit; in the
- 32-bit library it is a 32-bit unit. Unlike a dot, \C always matches
- line-ending characters. The feature is provided in Perl in order to
+ Outside a character class, the escape sequence \C matches any one code
+ unit, whether or not a UTF mode is set. In the 8-bit library, one code
+ unit is one byte; in the 16-bit library it is a 16-bit unit; in the
+ 32-bit library it is a 32-bit unit. Unlike a dot, \C always matches
+ line-ending characters. The feature is provided in Perl in order to
match individual bytes in UTF-8 mode, but it is unclear how it can use-
fully be used.
- Because \C breaks up characters into individual code units, matching
- one unit with \C in UTF-8 or UTF-16 mode means that the rest of the
+ Because \C breaks up characters into individual code units, matching
+ one unit with \C in UTF-8 or UTF-16 mode means that the rest of the
string may start with a malformed UTF character. This has undefined re-
sults, because PCRE2 assumes that it is matching character by character
in a valid UTF string (by default it checks the subject string's valid-
- ity at the start of processing unless the PCRE2_NO_UTF_CHECK or
+ ity at the start of processing unless the PCRE2_NO_UTF_CHECK or
PCRE2_MATCH_INVALID_UTF option is used).
- An application can lock out the use of \C by setting the
- PCRE2_NEVER_BACKSLASH_C option when compiling a pattern. It is also
+ An application can lock out the use of \C by setting the
+ PCRE2_NEVER_BACKSLASH_C option when compiling a pattern. It is also
possible to build PCRE2 with the use of \C permanently disabled.
- PCRE2 does not allow \C to appear in lookbehind assertions (described
- below) in UTF-8 or UTF-16 modes, because this would make it impossible
- to calculate the length of the lookbehind. Neither the alternative
+ PCRE2 does not allow \C to appear in lookbehind assertions (described
+ below) in UTF-8 or UTF-16 modes, because this would make it impossible
+ to calculate the length of the lookbehind. Neither the alternative
matching function pcre2_dfa_match() nor the JIT optimizer support \C in
these UTF modes. The former gives a match-time error; the latter fails
to optimize and so the match is always run using the interpreter.
- In the 32-bit library, however, \C is always supported (when not ex-
- plicitly locked out) because it always matches a single code unit,
+ In the 32-bit library, however, \C is always supported (when not ex-
+ plicitly locked out) because it always matches a single code unit,
whether or not UTF-32 is specified.
In general, the \C escape sequence is best avoided. However, one way of
- using it that avoids the problem of malformed UTF-8 or UTF-16 charac-
- ters is to use a lookahead to check the length of the next character,
- as in this pattern, which could be used with a UTF-8 string (ignore
+ using it that avoids the problem of malformed UTF-8 or UTF-16 charac-
+ ters is to use a lookahead to check the length of the next character,
+ as in this pattern, which could be used with a UTF-8 string (ignore
white space and line breaks):
(?| (?=[\x00-\x7f])(\C) |
@@ -8022,11 +8175,11 @@ MATCHING A SINGLE CODE UNIT
(?=[\x{800}-\x{ffff}])(\C)(\C)(\C) |
(?=[\x{10000}-\x{1fffff}])(\C)(\C)(\C)(\C))
- In this example, a group that starts with (?| resets the capturing
- parentheses numbers in each alternative (see "Duplicate Group Numbers"
+ In this example, a group that starts with (?| resets the capturing
+ parentheses numbers in each alternative (see "Duplicate Group Numbers"
below). The assertions at the start of each branch check the next UTF-8
- character for values whose encoding uses 1, 2, 3, or 4 bytes, respec-
- tively. The character's individual bytes are then captured by the ap-
+ character for values whose encoding uses 1, 2, 3, or 4 bytes, respec-
+ tively. The character's individual bytes are then captured by the ap-
propriate number of \C groups.
@@ -8034,223 +8187,223 @@ SQUARE BRACKETS AND CHARACTER CLASSES
An opening square bracket introduces a character class, terminated by a
closing square bracket. A closing square bracket on its own is not spe-
- cial by default. If a closing square bracket is required as a member
+ cial by default. If a closing square bracket is required as a member
of the class, it should be the first data character in the class (after
- an initial circumflex, if present) or escaped with a backslash. This
- means that, by default, an empty class cannot be defined. However, if
- the PCRE2_ALLOW_EMPTY_CLASS option is set, a closing square bracket at
+ an initial circumflex, if present) or escaped with a backslash. This
+ means that, by default, an empty class cannot be defined. However, if
+ the PCRE2_ALLOW_EMPTY_CLASS option is set, a closing square bracket at
the start does end the (empty) class.
- A character class matches a single character in the subject. A matched
+ A character class matches a single character in the subject. A matched
character must be in the set of characters defined by the class, unless
- the first character in the class definition is a circumflex, in which
+ the first character in the class definition is a circumflex, in which
case the subject character must not be in the set defined by the class.
- If a circumflex is actually required as a member of the class, ensure
+ If a circumflex is actually required as a member of the class, ensure
it is not the first character, or escape it with a backslash.
For example, the character class [aeiou] matches any lower case English
- vowel, whereas [^aeiou] matches all other characters. Note that a cir-
- cumflex is just a convenient notation for specifying the characters
- that are in the class by enumerating those that are not. A class that
+ vowel, whereas [^aeiou] matches all other characters. Note that a cir-
+ cumflex is just a convenient notation for specifying the characters
+ that are in the class by enumerating those that are not. A class that
starts with a circumflex is not an assertion; it still consumes a char-
- acter from the subject string, and therefore it fails to match if the
+ acter from the subject string, and therefore it fails to match if the
current pointer is at the end of the string.
- Characters in a class may be specified by their code points using \o,
- \x, or \N{U+hh..} in the usual way. When caseless matching is set, any
- letters in a class represent both their upper case and lower case ver-
- sions, so for example, a caseless [aeiou] matches "A" as well as "a",
- and a caseless [^aeiou] does not match "A", whereas a caseful version
- would. Note that there are two ASCII characters, K and S, that, in ad-
- dition to their lower case ASCII equivalents, are case-equivalent with
- Unicode U+212A (Kelvin sign) and U+017F (long S) respectively when ei-
+ Characters in a class may be specified by their code points using \o,
+ \x, or \N{U+hh..} in the usual way. When caseless matching is set, any
+ letters in a class represent both their upper case and lower case ver-
+ sions, so for example, a caseless [aeiou] matches "A" as well as "a",
+ and a caseless [^aeiou] does not match "A", whereas a caseful version
+ would. Note that there are two ASCII characters, K and S, that, in ad-
+ dition to their lower case ASCII equivalents, are case-equivalent with
+ Unicode U+212A (Kelvin sign) and U+017F (long S) respectively when ei-
ther PCRE2_UTF or PCRE2_UCP is set. If you do not want these ASCII/non-
- ASCII case equivalences, you can suppress them by setting PCRE2_EX-
- TRA_CASELESS_RESTRICT, either as an option in a compile context, or by
+ ASCII case equivalences, you can suppress them by setting PCRE2_EX-
+ TRA_CASELESS_RESTRICT, either as an option in a compile context, or by
including (*CASELESS_RESTRICT) or (?r) within a pattern.
- Characters that might indicate line breaks are never treated in any
- special way when matching character classes, whatever line-ending se-
- quence is in use, and whatever setting of the PCRE2_DOTALL and
- PCRE2_MULTILINE options is used. A class such as [^a] always matches
+ Characters that might indicate line breaks are never treated in any
+ special way when matching character classes, whatever line-ending se-
+ quence is in use, and whatever setting of the PCRE2_DOTALL and
+ PCRE2_MULTILINE options is used. A class such as [^a] always matches
one of these characters.
The generic character type escape sequences \d, \D, \h, \H, \p, \P, \s,
- \S, \v, \V, \w, and \W may appear in a character class, and add the
- characters that they match to the class. For example, [\dABCDEF]
- matches any hexadecimal digit. In UTF modes, the PCRE2_UCP option af-
+ \S, \v, \V, \w, and \W may appear in a character class, and add the
+ characters that they match to the class. For example, [\dABCDEF]
+ matches any hexadecimal digit. In UTF modes, the PCRE2_UCP option af-
fects the meanings of \d, \s, \w and their upper case partners, just as
it does when they appear outside a character class, as described in the
- section entitled "Generic character types" above. The escape sequence
- \b has a different meaning inside a character class; it matches the
- backspace character. The sequences \B, \R, and \X are not special in-
- side a character class. Like any other unrecognized escape sequences,
- they cause an error. The same is true for \N when not followed by an
+ section entitled "Generic character types" above. The escape sequence
+ \b has a different meaning inside a character class; it matches the
+ backspace character. The sequences \B, \R, and \X are not special in-
+ side a character class. Like any other unrecognized escape sequences,
+ they cause an error. The same is true for \N when not followed by an
opening brace.
- The minus (hyphen) character can be used to specify a range of charac-
- ters in a character class. For example, [d-m] matches any letter be-
- tween d and m, inclusive. If a minus character is required in a class,
- it must be escaped with a backslash or appear in a position where it
- cannot be interpreted as indicating a range, typically as the first or
+ The minus (hyphen) character can be used to specify a range of charac-
+ ters in a character class. For example, [d-m] matches any letter be-
+ tween d and m, inclusive. If a minus character is required in a class,
+ it must be escaped with a backslash or appear in a position where it
+ cannot be interpreted as indicating a range, typically as the first or
last character in the class, or immediately after a range. For example,
[b-d-z] matches letters in the range b to d, a hyphen character, or z.
- There is some special treatment for alphabetic ranges in EBCDIC envi-
+ There is some special treatment for alphabetic ranges in EBCDIC envi-
ronments; see the section "EBCDIC environments" below.
Perl treats a hyphen as a literal if it appears before or after a POSIX
class (see below) or before or after a character type escape such as \d
- or \H. However, unless the hyphen is the last character in the class,
- Perl outputs a warning in its warning mode, as this is most likely a
- user error. As PCRE2 has no facility for warning, an error is given in
+ or \H. However, unless the hyphen is the last character in the class,
+ Perl outputs a warning in its warning mode, as this is most likely a
+ user error. As PCRE2 has no facility for warning, an error is given in
these cases.
It is not possible to have the literal character "]" as the end charac-
- ter of a range. A pattern such as [W-]46] is interpreted as a class of
- two characters ("W" and "-") followed by a literal string "46]", so it
- would match "W46]" or "-46]". However, if the "]" is escaped with a
- backslash it is interpreted as the end of a range, so [W-\]46] is in-
- terpreted as a class containing a range and two other characters. The
- octal or hexadecimal representation of "]" can also be used to end a
+ ter of a range. A pattern such as [W-]46] is interpreted as a class of
+ two characters ("W" and "-") followed by a literal string "46]", so it
+ would match "W46]" or "-46]". However, if the "]" is escaped with a
+ backslash it is interpreted as the end of a range, so [W-\]46] is in-
+ terpreted as a class containing a range and two other characters. The
+ octal or hexadecimal representation of "]" can also be used to end a
range.
Ranges normally include all code points between the start and end char-
- acters, inclusive. They can also be used for code points specified nu-
- merically, for example [\000-\037]. Ranges can include any characters
- that are valid for the current mode. In any UTF mode, the so-called
- "surrogate" characters (those whose code points lie between 0xd800 and
- 0xdfff inclusive) may not be specified explicitly by default (the
- PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES option disables this check). How-
+ acters, inclusive. They can also be used for code points specified nu-
+ merically, for example [\000-\037]. Ranges can include any characters
+ that are valid for the current mode. In any UTF mode, the so-called
+ "surrogate" characters (those whose code points lie between 0xd800 and
+ 0xdfff inclusive) may not be specified explicitly by default (the
+ PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES option disables this check). How-
ever, ranges such as [\x{d7ff}-\x{e000}], which include the surrogates,
are always permitted.
If a range that includes letters is used when caseless matching is set,
it matches the letters in either case. For example, [W-c] is equivalent
- to [][\\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if
- character tables for a French locale are in use, [\xc8-\xcb] matches
+ to [][\\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if
+ character tables for a French locale are in use, [\xc8-\xcb] matches
accented E characters in both cases.
- A circumflex can conveniently be used with the upper case character
- types to specify a more restricted set of characters than the matching
- lower case type. For example, the class [^\W_] matches any letter or
+ A circumflex can conveniently be used with the upper case character
+ types to specify a more restricted set of characters than the matching
+ lower case type. For example, the class [^\W_] matches any letter or
digit, but not underscore, whereas [\w] includes underscore. A positive
character class should be read as "something OR something OR ..." and a
negative class as "NOT something AND NOT something AND NOT ...".
- The metacharacters that are recognized in character classes are back-
- slash, hyphen (when it can be interpreted as specifying a range), cir-
- cumflex (only at the start), and the terminating closing square
- bracket. An opening square bracket is also special when it can be in-
- terpreted as introducing a POSIX class (see "Posix character classes"
- below), or a special compatibility feature (see "Compatibility feature
- for word boundaries" below. Escaping any non-alphanumeric character in
+ The metacharacters that are recognized in character classes are back-
+ slash, hyphen (when it can be interpreted as specifying a range), cir-
+ cumflex (only at the start), and the terminating closing square
+ bracket. An opening square bracket is also special when it can be in-
+ terpreted as introducing a POSIX class (see "Posix character classes"
+ below), or a special compatibility feature (see "Compatibility feature
+ for word boundaries" below. Escaping any non-alphanumeric character in
a class turns it into a literal, whether or not it would otherwise be a
metacharacter.
PERL EXTENDED CHARACTER CLASSES
- From release 10.45 PCRE2 supports Perl's (?[...]) extended character
+ From release 10.45 PCRE2 supports Perl's (?[...]) extended character
class syntax. This can be used to perform set operations such as inter-
section on character classes.
- The syntax permitted within (?[...]) is quite different to ordinary
- character classes. Inside the extended class, there is an expression
- syntax consisting of "atoms", operators, and ordinary parentheses "()"
- used for grouping. Such classes always have the Perl /xx modifier
- (PCRE2 option PCRE2_EXTENDED_MORE) turned on within them. This means
- that literal space and tab characters are ignored everywhere in the
+ The syntax permitted within (?[...]) is quite different to ordinary
+ character classes. Inside the extended class, there is an expression
+ syntax consisting of "atoms", operators, and ordinary parentheses "()"
+ used for grouping. Such classes always have the Perl /xx modifier
+ (PCRE2 option PCRE2_EXTENDED_MORE) turned on within them. This means
+ that literal space and tab characters are ignored everywhere in the
class.
- The allowed atoms are individual characters specified by escape se-
- quences such as \n or \x{123}, character types such as \d, POSIX
+ The allowed atoms are individual characters specified by escape se-
+ quences such as \n or \x{123}, character types such as \d, POSIX
classes such as [:alpha:], and nested ordinary (non-extended) character
classes. For example, in (?[\d & [...]]) the nested class [...] follows
- the usual rules for ordinary character classes, in which parentheses
- are not metacharacters, and character literals and ranges are permit-
+ the usual rules for ordinary character classes, in which parentheses
+ are not metacharacters, and character literals and ranges are permit-
ted.
- Character literals and ranges may not appear outside a nested ordinary
- character class because they are not atoms in the extended syntax. The
- extended syntax does not introduce any additional escape sequences, so
+ Character literals and ranges may not appear outside a nested ordinary
+ character class because they are not atoms in the extended syntax. The
+ extended syntax does not introduce any additional escape sequences, so
(?[\y]) is an unknown escape, as it would be in [\y].
In the extended syntax, ^ does not negate a class (except within an or-
- dinary class nested inside an extended class); it is instead a binary
+ dinary class nested inside an extended class); it is instead a binary
operator.
- The binary operators are "&" (intersection), "|" or "+" (union), "-"
- (subtraction) and "^" (symmetric difference). These are left-associa-
- tive and "&" has higher (tighter) precedence, while the others have
- equal lower precedence. The one prefix unary operator is "!" (comple-
+ The binary operators are "&" (intersection), "|" or "+" (union), "-"
+ (subtraction) and "^" (symmetric difference). These are left-associa-
+ tive and "&" has higher (tighter) precedence, while the others have
+ equal lower precedence. The one prefix unary operator is "!" (comple-
ment), with highest precedence.
UTS#18 EXTENDED CHARACTER CLASSES
- The PCRE2_ALT_EXTENDED_CLASS option enables an alternative to Perl's
- (?[...]) syntax, allowing instead extended class behaviour inside or-
- dinary [...] character classes. This altered syntax for [...] classes
- is loosely described by the Unicode standard UTS#18. The PCRE2_ALT_EX-
- TENDED_CLASS option does not prevent use of (?[...]) classes; it just
- changes the meaning of all [...] classes that are not nested inside a
+ The PCRE2_ALT_EXTENDED_CLASS option enables an alternative to Perl's
+ (?[...]) syntax, allowing instead extended class behaviour inside or-
+ dinary [...] character classes. This altered syntax for [...] classes
+ is loosely described by the Unicode standard UTS#18. The PCRE2_ALT_EX-
+ TENDED_CLASS option does not prevent use of (?[...]) classes; it just
+ changes the meaning of all [...] classes that are not nested inside a
Perl (?[...]) class.
Firstly, in ordinary Perl [...] syntax, an expression such as "[a[]" is
- a character class with two literal characters "a" and "[", but in
+ a character class with two literal characters "a" and "[", but in
UTS#18 extended classes the "[" character becomes an additional
- metacharacter within classes, denoting the start of a nested class, so
+ metacharacter within classes, denoting the start of a nested class, so
a literal "[" must be escaped as "\[".
- Secondly, within the UTS#18 extended syntax, there are operators "||",
- "&&", "--" and "~~" which denote character class union, intersection,
- subtraction, and symmetric difference respectively. In standard Perl
- syntax, these would simply be needlessly-repeated literals (except for
- "--" which could be the start or end of a range). In UTS#18 extended
+ Secondly, within the UTS#18 extended syntax, there are operators "||",
+ "&&", "--" and "~~" which denote character class union, intersection,
+ subtraction, and symmetric difference respectively. In standard Perl
+ syntax, these would simply be needlessly-repeated literals (except for
+ "--" which could be the start or end of a range). In UTS#18 extended
classes these operators can be used in constructs such as [\p{L}--[QW]]
- for "Unicode letters, other than Q and W". A literal "-" at the start
- or end of a range must be escaped, so while "[--1]" in Perl syntax is
- the range from hyphen to "1", it must be escaped as "[\--1]" in UTS#18
+ for "Unicode letters, other than Q and W". A literal "-" at the start
+ or end of a range must be escaped, so while "[--1]" in Perl syntax is
+ the range from hyphen to "1", it must be escaped as "[\--1]" in UTS#18
extended classes.
Unlike Perl's (?[...]) extended classes, the PCRE2_EXTENDED_MORE option
- to ignore space and tab characters is not automatically enabled for
+ to ignore space and tab characters is not automatically enabled for
UTS#18 extended classes, but it is honoured if set.
- Extended UTS#18 classes can be nested, and nested classes are them-
+ Extended UTS#18 classes can be nested, and nested classes are them-
selves extended classes (unlike Perl, where nested classes must be sim-
- ple classes). For example, [\p{L}&&[\p{Thai}||\p{Greek}]] matches any
- letter that is in the Thai or Greek scripts. Note that this means that
- no special grouping characters (such as the parentheses used in Perl's
+ ple classes). For example, [\p{L}&&[\p{Thai}||\p{Greek}]] matches any
+ letter that is in the Thai or Greek scripts. Note that this means that
+ no special grouping characters (such as the parentheses used in Perl's
(?[...]) class syntax) are needed.
- Individual class items (literal characters, literal ranges, properties
- such as \d or \p{...}, and nested classes) can be combined by juxtapo-
+ Individual class items (literal characters, literal ranges, properties
+ such as \d or \p{...}, and nested classes) can be combined by juxtapo-
sition or by an operator. Juxtaposition is the implicit union operator,
- and binds more tightly than any explicit operator. Thus a sequence of
+ and binds more tightly than any explicit operator. Thus a sequence of
literals and/or ranges behaves as if it is enclosed in square brackets.
- For example, [A-Z0-9&&[^E8]] is the same as [[A-Z0-9]&&[^E8]], which
+ For example, [A-Z0-9&&[^E8]] is the same as [[A-Z0-9]&&[^E8]], which
matches any upper case alphanumeric character except "E" or "8".
Precedence between the explicit operators is not defined, so mixing op-
- erators is a syntax error. For example, [A&&B--C] is an error, but
+ erators is a syntax error. For example, [A&&B--C] is an error, but
[A&&[B--C]] is valid.
- This is an emerging syntax which is being adopted gradually across the
- regex ecosystem: for example JavaScript adopted the "/v" flag in EC-
- MAScript 2024; Python's "re" module reserves the syntax for future use
+ This is an emerging syntax which is being adopted gradually across the
+ regex ecosystem: for example JavaScript adopted the "/v" flag in EC-
+ MAScript 2024; Python's "re" module reserves the syntax for future use
with a FutureWarning for unescaped use of "[" as a literal within char-
- acter classes. Due to UTS#18 providing insufficient guidance, engines
- interpret the syntax differently. Rust's "regex" crate and Python's
- "regex" PyPi module both implement UTS#18 extended classes, but with
- slight incompatibilities ([A||B&&C] is parsed as [A||[B&&C]] in
+ acter classes. Due to UTS#18 providing insufficient guidance, engines
+ interpret the syntax differently. Rust's "regex" crate and Python's
+ "regex" PyPi module both implement UTS#18 extended classes, but with
+ slight incompatibilities ([A||B&&C] is parsed as [A||[B&&C]] in
Python's "regex" but as [[A||B]&&C] in Rust's "regex").
- PCRE2's syntax adds syntax restrictions similar to ECMASCript's /v
- flag, so that all the UTS#18 extended classes accepted as valid by
- PCRE2 have the property that they are interpreted either with the same
- behaviour, or as invalid, by all other major engines. Please file an
+ PCRE2's syntax adds syntax restrictions similar to ECMASCript's /v
+ flag, so that all the UTS#18 extended classes accepted as valid by
+ PCRE2 have the property that they are interpreted either with the same
+ behaviour, or as invalid, by all other major engines. Please file an
issue if you are aware of cross-engine differences in behaviour between
PCRE2 and another major engine.
@@ -8258,8 +8411,8 @@ UTS#18 EXTENDED CHARACTER CLASSES
POSIX CHARACTER CLASSES
Perl supports the POSIX notation for character classes. This uses names
- enclosed by [: and :] within the enclosing square brackets. PCRE2 also
- supports this notation, in both ordinary and extended classes. For ex-
+ enclosed by [: and :] within the enclosing square brackets. PCRE2 also
+ supports this notation, in both ordinary and extended classes. For ex-
ample,
[01[:alpha:]%]
@@ -8282,14 +8435,14 @@ POSIX CHARACTER CLASSES
word "word" characters (same as \w)
xdigit hexadecimal digits
- The default "space" characters are HT (9), LF (10), VT (11), FF (12),
- CR (13), and space (32). If locale-specific matching is taking place,
- the list of space characters may be different; there may be fewer or
- more of them. "Space" and \s match the same set of characters, as do
+ The default "space" characters are HT (9), LF (10), VT (11), FF (12),
+ CR (13), and space (32). If locale-specific matching is taking place,
+ the list of space characters may be different; there may be fewer or
+ more of them. "Space" and \s match the same set of characters, as do
"word" and \w.
- The name "word" is a Perl extension, and "blank" is a GNU extension
- from Perl 5.8. Another Perl extension is negation, which is indicated
+ The name "word" is a Perl extension, and "blank" is a GNU extension
+ from Perl 5.8. Another Perl extension is negation, which is indicated
by a ^ character after the colon. For example,
[12[:^digit:]]
@@ -8300,9 +8453,9 @@ POSIX CHARACTER CLASSES
By default, characters with values greater than 127 do not match any of
the POSIX character classes, although this may be different for charac-
- ters in the range 128-255 when locale-specific matching is happening.
- However, in UCP mode, unless certain options are set (see below), some
- of the classes are changed so that Unicode character properties are
+ ters in the range 128-255 when locale-specific matching is happening.
+ However, in UCP mode, unless certain options are set (see below), some
+ of the classes are changed so that Unicode character properties are
used. This is achieved by replacing POSIX classes with other sequences,
as follows:
@@ -8316,10 +8469,10 @@ POSIX CHARACTER CLASSES
[:upper:] becomes \p{Lu}
[:word:] becomes \p{Xwd}
- Negated versions, such as [:^alpha:] use \P instead of \p. Four other
+ Negated versions, such as [:^alpha:] use \P instead of \p. Four other
POSIX classes are handled specially in UCP mode:
- [:graph:] This matches characters that have glyphs that mark the page
+ [:graph:] This matches characters that have glyphs that mark the page
when printed. In Unicode property terms, it matches all char-
acters with the L, M, N, P, S, or Cf properties, except for:
@@ -8328,73 +8481,73 @@ POSIX CHARACTER CLASSES
U+2066 - U+2069 Various "isolate"s
- [:print:] This matches the same characters as [:graph:] plus space
- characters that are not controls, that is, characters with
+ [:print:] This matches the same characters as [:graph:] plus space
+ characters that are not controls, that is, characters with
the Zs property.
[:punct:] This matches all characters that have the Unicode P (punctua-
- tion) property, plus those characters with code points less
+ tion) property, plus those characters with code points less
than 256 that have the S (Symbol) property.
[:xdigit:]
- In addition to the ASCII hexadecimal digits, this also
- matches the "fullwidth" versions of those characters, whose
- Unicode code points start at U+FF10. This is a change that
+ In addition to the ASCII hexadecimal digits, this also
+ matches the "fullwidth" versions of those characters, whose
+ Unicode code points start at U+FF10. This is a change that
was made in PCRE2 release 10.43 for Perl compatibility.
- The other POSIX classes are unchanged by PCRE2_UCP, and match only
+ The other POSIX classes are unchanged by PCRE2_UCP, and match only
characters with code points less than 256.
There are two options that can be used to restrict the POSIX classes to
- ASCII characters when PCRE2_UCP is set. The option PCRE2_EX-
- TRA_ASCII_DIGIT affects just [:digit:] and [:xdigit:]. Within a pat-
- tern, this can be set and unset by (?aT) and (?-aT). The PCRE2_EX-
- TRA_ASCII_POSIX option disables UCP processing for all POSIX classes,
- including [:digit:] and [:xdigit:]. Within a pattern, (?aP) and (?-aP)
+ ASCII characters when PCRE2_UCP is set. The option PCRE2_EX-
+ TRA_ASCII_DIGIT affects just [:digit:] and [:xdigit:]. Within a pat-
+ tern, this can be set and unset by (?aT) and (?-aT). The PCRE2_EX-
+ TRA_ASCII_POSIX option disables UCP processing for all POSIX classes,
+ including [:digit:] and [:xdigit:]. Within a pattern, (?aP) and (?-aP)
set and unset both these options for consistency.
COMPATIBILITY FEATURE FOR WORD BOUNDARIES
- In the POSIX.2 compliant library that was included in 4.4BSD Unix, the
- ugly syntax [[:<:]] and [[:>:]] is used for matching "start of word"
+ In the POSIX.2 compliant library that was included in 4.4BSD Unix, the
+ ugly syntax [[:<:]] and [[:>:]] is used for matching "start of word"
and "end of word". PCRE2 treats these items as follows:
[[:<:]] is converted to \b(?=\w)
[[:>:]] is converted to \b(?<=\w)
Only these exact character sequences are recognized. A sequence such as
- [a[:<:]b] provokes error for an unrecognized POSIX class name. This
- support is not compatible with Perl. It is provided to help migrations
+ [a[:<:]b] provokes error for an unrecognized POSIX class name. This
+ support is not compatible with Perl. It is provided to help migrations
from other environments, and is best not used in any new patterns. Note
- that \b matches at the start and the end of a word (see "Simple asser-
- tions" above), and in a Perl-style pattern the preceding or following
- character normally shows which is wanted, without the need for the as-
- sertions that are used above in order to give exactly the POSIX behav-
- iour. Note also that the PCRE2_UCP option changes the meaning of \w
- (and therefore \b) by default, so it also affects these POSIX se-
+ that \b matches at the start and the end of a word (see "Simple asser-
+ tions" above), and in a Perl-style pattern the preceding or following
+ character normally shows which is wanted, without the need for the as-
+ sertions that are used above in order to give exactly the POSIX behav-
+ iour. Note also that the PCRE2_UCP option changes the meaning of \w
+ (and therefore \b) by default, so it also affects these POSIX se-
quences.
VERTICAL BAR
- Vertical bar characters are used to separate alternative patterns. For
+ Vertical bar characters are used to separate alternative patterns. For
example, the pattern
gilbert|sullivan
- matches either "gilbert" or "sullivan". Any number of alternatives may
- appear, and an empty alternative is permitted (matching the empty
+ matches either "gilbert" or "sullivan". Any number of alternatives may
+ appear, and an empty alternative is permitted (matching the empty
string). The matching process tries each alternative in turn, from left
- to right, and the first one that succeeds is used. If the alternatives
- are within a group (defined below), "succeeds" means matching the rest
+ to right, and the first one that succeeds is used. If the alternatives
+ are within a group (defined below), "succeeds" means matching the rest
of the main pattern as well as the alternative in the group.
INTERNAL OPTION SETTING
- The settings of several options can be changed within a pattern by a
- sequence of letters enclosed between "(?" and ")". The following are
+ The settings of several options can be changed within a pattern by a
+ sequence of letters enclosed between "(?" and ")". The following are
Perl-compatible, and are described in detail in the pcre2api documenta-
tion. The option letters are:
@@ -8407,21 +8560,21 @@ INTERNAL OPTION SETTING
For example, (?im) sets caseless, multiline matching. It is also possi-
ble to unset these options by preceding the relevant letters with a hy-
- phen, for example (?-im). The two "extended" options are not indepen-
+ phen, for example (?-im). The two "extended" options are not indepen-
dent; unsetting either one cancels the effects of both of them.
- A combined setting and unsetting such as (?im-sx), which sets
- PCRE2_CASELESS and PCRE2_MULTILINE while unsetting PCRE2_DOTALL and
- PCRE2_EXTENDED, is also permitted. Only one hyphen may appear in the
- options string. If a letter appears both before and after the hyphen,
- the option is unset. An empty options setting "(?)" is allowed. Need-
+ A combined setting and unsetting such as (?im-sx), which sets
+ PCRE2_CASELESS and PCRE2_MULTILINE while unsetting PCRE2_DOTALL and
+ PCRE2_EXTENDED, is also permitted. Only one hyphen may appear in the
+ options string. If a letter appears both before and after the hyphen,
+ the option is unset. An empty options setting "(?)" is allowed. Need-
less to say, it has no effect.
- If the first character following (? is a circumflex, it causes all of
- the above options to be unset. Letters may follow the circumflex to
+ If the first character following (? is a circumflex, it causes all of
+ the above options to be unset. Letters may follow the circumflex to
cause some options to be re-instated, but a hyphen may not appear.
- Some PCRE2-specific options can be changed by the same mechanism using
+ Some PCRE2-specific options can be changed by the same mechanism using
these pairs or individual letters:
aD for PCRE2_EXTRA_ASCII_BSD
@@ -8433,36 +8586,36 @@ INTERNAL OPTION SETTING
J for PCRE2_DUPNAMES
U for PCRE2_UNGREEDY
- However, except for 'r', these are not unset by (?^), which is equiva-
- lent to (?-imnrsx). If 'a' is not followed by any of the upper case
+ However, except for 'r', these are not unset by (?^), which is equiva-
+ lent to (?-imnrsx). If 'a' is not followed by any of the upper case
letters shown above, it sets (or unsets) all the ASCII options.
- PCRE2_EXTRA_ASCII_DIGIT has no additional effect when PCRE2_EX-
- TRA_ASCII_POSIX is set, but including it in (?aP) means that (?-aP)
+ PCRE2_EXTRA_ASCII_DIGIT has no additional effect when PCRE2_EX-
+ TRA_ASCII_POSIX is set, but including it in (?aP) means that (?-aP)
suppresses all ASCII restrictions for POSIX classes.
- When one of these option changes occurs at top level (that is, not in-
- side group parentheses), the change applies until a subsequent change,
- or the end of the pattern. An option change within a group (see below
- for a description of groups) affects only that part of the group that
- follows it. At the end of the group these options are reset to the
+ When one of these option changes occurs at top level (that is, not in-
+ side group parentheses), the change applies until a subsequent change,
+ or the end of the pattern. An option change within a group (see below
+ for a description of groups) affects only that part of the group that
+ follows it. At the end of the group these options are reset to the
state they were before the group. For example,
(a(?i)b)c
- matches abc and aBc and no other strings (assuming PCRE2_CASELESS is
- not set externally). Any changes made in one alternative do carry on
+ matches abc and aBc and no other strings (assuming PCRE2_CASELESS is
+ not set externally). Any changes made in one alternative do carry on
into subsequent branches within the same group. For example,
(a(?i)b|c)
- matches "ab", "aB", "c", and "C", even though when matching "C" the
- first branch is abandoned before the option setting. This is because
- the effects of option settings happen at compile time. There would be
+ matches "ab", "aB", "c", and "C", even though when matching "C" the
+ first branch is abandoned before the option setting. This is because
+ the effects of option settings happen at compile time. There would be
some very weird behaviour otherwise.
- As a convenient shorthand, if any option settings are required at the
- start of a non-capturing group (see the next section), the option let-
+ As a convenient shorthand, if any option settings are required at the
+ start of a non-capturing group (see the next section), the option let-
ters may appear between the "?" and the ":". Thus the two patterns
(?i:saturday|sunday)
@@ -8470,39 +8623,39 @@ INTERNAL OPTION SETTING
match exactly the same set of strings.
- Note: There are other PCRE2-specific options, applying to the whole
- pattern, which can be set by the application when the compiling func-
- tion is called. In addition, the pattern can contain special leading
- sequences such as (*CRLF) to override what the application has set or
- what has been defaulted. Details are given in the section entitled
+ Note: There are other PCRE2-specific options, applying to the whole
+ pattern, which can be set by the application when the compiling func-
+ tion is called. In addition, the pattern can contain special leading
+ sequences such as (*CRLF) to override what the application has set or
+ what has been defaulted. Details are given in the section entitled
"Newline sequences" above. There are also the (*UTF) and (*UCP) leading
- sequences that can be used to set UTF and Unicode property modes; they
- are equivalent to setting the PCRE2_UTF and PCRE2_UCP options, respec-
+ sequences that can be used to set UTF and Unicode property modes; they
+ are equivalent to setting the PCRE2_UTF and PCRE2_UCP options, respec-
tively. However, the application can set the PCRE2_NEVER_UTF or
- PCRE2_NEVER_UCP options, which lock out the use of the (*UTF) and
+ PCRE2_NEVER_UCP options, which lock out the use of the (*UTF) and
(*UCP) sequences.
GROUPS
- Groups are delimited by parentheses (round brackets), which can be
+ Groups are delimited by parentheses (round brackets), which can be
nested. Turning part of a pattern into a group does two things:
1. It localizes a set of alternatives. For example, the pattern
cat(aract|erpillar|)
- matches "cataract", "caterpillar", or "cat". Without the parentheses,
+ matches "cataract", "caterpillar", or "cat". Without the parentheses,
it would match "cataract", "erpillar" or an empty string.
- 2. It creates a "capture group". This means that, when the whole pat-
- tern matches, the portion of the subject string that matched the group
- is passed back to the caller, separately from the portion that matched
- the whole pattern. (This applies only to the traditional matching
+ 2. It creates a "capture group". This means that, when the whole pat-
+ tern matches, the portion of the subject string that matched the group
+ is passed back to the caller, separately from the portion that matched
+ the whole pattern. (This applies only to the traditional matching
function; the DFA matching function does not support capturing.)
Opening parentheses are counted from left to right (starting from 1) to
- obtain numbers for capture groups. For example, if the string "the red
+ obtain numbers for capture groups. For example, if the string "the red
king" is matched against the pattern
the ((red|white) (king|queen))
@@ -8510,11 +8663,11 @@ GROUPS
the captured substrings are "red king", "red", and "king", and are num-
bered 1, 2, and 3, respectively.
- The fact that plain parentheses fulfil two functions is not always
- helpful. There are often times when grouping is required without cap-
- turing. If an opening parenthesis is followed by a question mark and a
- colon, the group does not do any capturing, and is not counted when
- computing the number of any subsequent capture groups. For example, if
+ The fact that plain parentheses fulfil two functions is not always
+ helpful. There are often times when grouping is required without cap-
+ turing. If an opening parenthesis is followed by a question mark and a
+ colon, the group does not do any capturing, and is not counted when
+ computing the number of any subsequent capture groups. For example, if
the string "the white queen" is matched against the pattern
the ((?:red|white) (king|queen))
@@ -8522,16 +8675,16 @@ GROUPS
the captured substrings are "white queen" and "queen", and are numbered
1 and 2. The maximum number of capture groups is 65535.
- As a convenient shorthand, if any option settings are required at the
- start of a non-capturing group, the option letters may appear between
+ As a convenient shorthand, if any option settings are required at the
+ start of a non-capturing group, the option letters may appear between
the "?" and the ":". Thus the two patterns
(?i:saturday|sunday)
(?:(?i)saturday|sunday)
match exactly the same set of strings. Because alternative branches are
- tried from left to right, and options are not reset until the end of
- the group is reached, an option setting in one branch does affect sub-
+ tried from left to right, and options are not reset until the end of
+ the group is reached, an option setting in one branch does affect sub-
sequent branches, so the above patterns match "SUNDAY" as well as "Sat-
urday".
@@ -8539,19 +8692,19 @@ GROUPS
DUPLICATE GROUP NUMBERS
Perl 5.10 introduced a feature whereby each alternative in a group uses
- the same numbers for its capturing parentheses. Such a group starts
- with (?| and is itself a non-capturing group. For example, consider
+ the same numbers for its capturing parentheses. Such a group starts
+ with (?| and is itself a non-capturing group. For example, consider
this pattern:
(?|(Sat)ur|(Sun))day
- Because the two alternatives are inside a (?| group, both sets of cap-
- turing parentheses are numbered one. Thus, when the pattern matches,
- you can look at captured substring number one, whichever alternative
- matched. This construct is useful when you want to capture part, but
+ Because the two alternatives are inside a (?| group, both sets of cap-
+ turing parentheses are numbered one. Thus, when the pattern matches,
+ you can look at captured substring number one, whichever alternative
+ matched. This construct is useful when you want to capture part, but
not all, of one of a number of alternatives. Inside a (?| group, paren-
- theses are numbered as usual, but the number is reset at the start of
- each branch. The numbers of any capturing parentheses that follow the
+ theses are numbered as usual, but the number is reset at the start of
+ each branch. The numbers of any capturing parentheses that follow the
whole group start after the highest number used in any branch. The fol-
lowing example is taken from the Perl documentation. The numbers under-
neath show in which buffer the captured content will be stored.
@@ -8560,13 +8713,13 @@ DUPLICATE GROUP NUMBERS
/ ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x
# 1 2 2 3 2 3 4
- A backreference to a capture group uses the most recent value that is
+ A backreference to a capture group uses the most recent value that is
set for the group. The following pattern matches "abcabc" or "defdef":
/(?|(abc)|(def))\1/
- In contrast, a subroutine call to a capture group always refers to the
- first one in the pattern with the given number. The following pattern
+ In contrast, a subroutine call to a capture group always refers to the
+ first one in the pattern with the given number. The following pattern
matches "abcabc" or "defabc":
/(?|(abc)|(def))(?1)/
@@ -8577,24 +8730,24 @@ DUPLICATE GROUP NUMBERS
If a condition test for a group's having matched refers to a non-unique
number, the test is true if any group with that number has matched.
- An alternative approach to using this "branch reset" feature is to use
+ An alternative approach to using this "branch reset" feature is to use
duplicate named groups, as described in the next section.
NAMED CAPTURE GROUPS
Identifying capture groups by number is simple, but it can be very hard
- to keep track of the numbers in complicated patterns. Furthermore, if
- an expression is modified, the numbers may change. To help with this
- difficulty, PCRE2 supports the naming of capture groups. This feature
- was not added to Perl until release 5.10. Python had the feature ear-
- lier, and PCRE1 introduced it at release 4.0, using the Python syntax.
+ to keep track of the numbers in complicated patterns. Furthermore, if
+ an expression is modified, the numbers may change. To help with this
+ difficulty, PCRE2 supports the naming of capture groups. This feature
+ was not added to Perl until release 5.10. Python had the feature ear-
+ lier, and PCRE1 introduced it at release 4.0, using the Python syntax.
PCRE2 supports both the Perl and the Python syntax.
- In PCRE2, a capture group can be named in one of three ways:
+ In PCRE2, a capture group can be named in one of three ways:
(?...) or (?'name'...) as in Perl, or (?P...) as in Python.
Names may be up to 128 code units long. When PCRE2_UTF is not set, they
- may contain only ASCII alphanumeric characters and underscores, but
+ may contain only ASCII alphanumeric characters and underscores, but
must start with a non-digit. When PCRE2_UTF is set, the syntax of group
names is extended to allow any Unicode letter or Unicode decimal digit.
In other words, group names must match one of these patterns:
@@ -8602,42 +8755,42 @@ NAMED CAPTURE GROUPS
^[_A-Za-z][_A-Za-z0-9]*\z when PCRE2_UTF is not set
^[_\p{L}][_\p{L}\p{Nd}]*\z when PCRE2_UTF is set
- References to capture groups from other parts of the pattern, such as
- backreferences, recursion, and conditions, can all be made by name as
+ References to capture groups from other parts of the pattern, such as
+ backreferences, recursion, and conditions, can all be made by name as
well as by number.
Named capture groups are allocated numbers as well as names, exactly as
- if the names were not present. In both PCRE2 and Perl, capture groups
- are primarily identified by numbers; any names are just aliases for
+ if the names were not present. In both PCRE2 and Perl, capture groups
+ are primarily identified by numbers; any names are just aliases for
these numbers. The PCRE2 API provides function calls for extracting the
- complete name-to-number translation table from a compiled pattern, as
- well as convenience functions for extracting captured substrings by
+ complete name-to-number translation table from a compiled pattern, as
+ well as convenience functions for extracting captured substrings by
name.
- Warning: When more than one capture group has the same number, as de-
+ Warning: When more than one capture group has the same number, as de-
scribed in the previous section, a name given to one of them applies to
- all of them. Perl allows identically numbered groups to have different
+ all of them. Perl allows identically numbered groups to have different
names. Consider this pattern, where there are two capture groups, both
numbered 1:
(?|(?aa)|(?bb))
- Perl allows this, with both names AA and BB as aliases of group 1.
+ Perl allows this, with both names AA and BB as aliases of group 1.
Thus, after a successful match, both names yield the same value (either
"aa" or "bb").
- In an attempt to reduce confusion, PCRE2 does not allow the same group
+ In an attempt to reduce confusion, PCRE2 does not allow the same group
number to be associated with more than one name. The example above pro-
- vokes a compile-time error. However, there is still scope for confu-
+ vokes a compile-time error. However, there is still scope for confu-
sion. Consider this pattern:
(?|(?aa)|(bb))
Although the second group number 1 is not explicitly named, the name AA
- is still an alias for any group 1. Whether the pattern matches "aa" or
+ is still an alias for any group 1. Whether the pattern matches "aa" or
"bb", a reference by name to group AA yields the matched string.
- By default, a name must be unique within a pattern, except that dupli-
+ By default, a name must be unique within a pattern, except that dupli-
cate names are permitted for groups with the same number, for example:
(?|(?aa)|(?bb))
@@ -8646,10 +8799,10 @@ NAMED CAPTURE GROUPS
NAMES option at compile time, or by the use of (?J) within the pattern,
as described in the section entitled "Internal Option Setting" above.
- Duplicate names can be useful for patterns where only one instance of
- the named capture group can match. Suppose you want to match the name
- of a weekday, either as a 3-letter abbreviation or as the full name,
- and in both cases you want to extract the abbreviation. This pattern
+ Duplicate names can be useful for patterns where only one instance of
+ the named capture group can match. Suppose you want to match the name
+ of a weekday, either as a 3-letter abbreviation or as the full name,
+ and in both cases you want to extract the abbreviation. This pattern
(ignoring the line breaks) does the job:
(?J)
@@ -8659,17 +8812,17 @@ NAMED CAPTURE GROUPS
(?Thu)(?:rsday)?|
(?Sat)(?:urday)?
- There are five capture groups, but only one is ever set after a match.
- The convenience functions for extracting the data by name returns the
- substring for the first (and in this example, the only) group of that
+ There are five capture groups, but only one is ever set after a match.
+ The convenience functions for extracting the data by name returns the
+ substring for the first (and in this example, the only) group of that
name that matched. This saves searching to find which numbered group it
- was. (An alternative way of solving this problem is to use a "branch
+ was. (An alternative way of solving this problem is to use a "branch
reset" group, as described in the previous section.)
- If you make a backreference to a non-unique named group from elsewhere
- in the pattern, the groups to which the name refers are checked in the
- order in which they appear in the overall pattern. The first one that
- is set is used for the reference. For example, this pattern matches
+ If you make a backreference to a non-unique named group from elsewhere
+ in the pattern, the groups to which the name refers are checked in the
+ order in which they appear in the overall pattern. The first one that
+ is set is used for the reference. For example, this pattern matches
both "foofoo" and "barbar" but not "foobar" or "barfoo":
(?J)(?:(?foo)|(?bar))\k
@@ -8682,15 +8835,15 @@ NAMED CAPTURE GROUPS
If you use a named reference in a condition test (see the section about
conditions below), either to check whether a capture group has matched,
or to check for recursion, all groups with the same name are tested. If
- the condition is true for any one of them, the overall condition is
- true. This is the same behaviour as testing by number. For further de-
- tails of the interfaces for handling named capture groups, see the
+ the condition is true for any one of them, the overall condition is
+ true. This is the same behaviour as testing by number. For further de-
+ tails of the interfaces for handling named capture groups, see the
pcre2api documentation.
REPETITION
- Repetition is specified by quantifiers, which may follow any one of
+ Repetition is specified by quantifiers, which may follow any one of
these items:
a literal data character
@@ -8706,16 +8859,16 @@ REPETITION
If a quantifier does not follow a repeatable item, an error occurs. The
general repetition quantifier specifies a minimum and maximum number of
- permitted matches by giving two numbers in curly brackets (braces),
- separated by a comma. The numbers must be less than 65536, and the
+ permitted matches by giving two numbers in curly brackets (braces),
+ separated by a comma. The numbers must be less than 65536, and the
first must be less than or equal to the second. For example,
z{2,4}
- matches "zz", "zzz", or "zzzz". A closing brace on its own is not a
- special character. If the second number is omitted, but the comma is
- present, there is no upper limit; if the second number and the comma
- are both omitted, the quantifier specifies an exact number of required
+ matches "zz", "zzz", or "zzzz". A closing brace on its own is not a
+ special character. If the second number is omitted, but the comma is
+ present, there is no upper limit; if the second number and the comma
+ are both omitted, the quantifier specifies an exact number of required
matches. Thus
[aeiou]{3,}
@@ -8724,65 +8877,65 @@ REPETITION
\d{8}
- matches exactly 8 digits. If the first number is omitted, the lower
+ matches exactly 8 digits. If the first number is omitted, the lower
limit is taken as zero; in this case the upper limit must be present.
X{,4} is interpreted as X{0,4}
- This is a change in behaviour that happened in Perl 5.34.0 and PCRE2
- 10.43. In earlier versions such a sequence was not interpreted as a
+ This is a change in behaviour that happened in Perl 5.34.0 and PCRE2
+ 10.43. In earlier versions such a sequence was not interpreted as a
quantifier. Other regular expression engines may behave either way.
- If the characters that follow an opening brace do not match the syntax
+ If the characters that follow an opening brace do not match the syntax
of a quantifier, the brace is taken as a literal character. In particu-
lar, this means that {,} is a literal string of three characters.
Note that not every opening brace is potentially the start of a quanti-
- fier because braces are used in other items such as \N{U+345} or
+ fier because braces are used in other items such as \N{U+345} or
\k{name}.
In UTF modes, quantifiers apply to characters rather than to individual
- code units. Thus, for example, \x{100}{2} matches two characters, each
+ code units. Thus, for example, \x{100}{2} matches two characters, each
of which is represented by a two-byte sequence in a UTF-8 string. Simi-
- larly, \X{3} matches three Unicode extended grapheme clusters, each of
- which may be several code units long (and they may be of different
+ larly, \X{3} matches three Unicode extended grapheme clusters, each of
+ which may be several code units long (and they may be of different
lengths).
The quantifier {0} is permitted, causing the expression to behave as if
the previous item and the quantifier were not present. This may be use-
- ful for capture groups that are referenced as subroutines from else-
- where in the pattern (but see also the section entitled "Defining cap-
+ ful for capture groups that are referenced as subroutines from else-
+ where in the pattern (but see also the section entitled "Defining cap-
ture groups for use by reference only" below). Except for parenthesized
- groups, items that have a {0} quantifier are omitted from the compiled
+ groups, items that have a {0} quantifier are omitted from the compiled
pattern.
- For convenience, the three most common quantifiers have single-charac-
+ For convenience, the three most common quantifiers have single-charac-
ter abbreviations:
* is equivalent to {0,}
+ is equivalent to {1,}
? is equivalent to {0,1}
- It is possible to construct infinite loops by following a group that
- can match no characters with a quantifier that has no upper limit, for
+ It is possible to construct infinite loops by following a group that
+ can match no characters with a quantifier that has no upper limit, for
example:
(a?)*
- Earlier versions of Perl and PCRE1 used to give an error at compile
+ Earlier versions of Perl and PCRE1 used to give an error at compile
time for such patterns. However, because there are cases where this can
be useful, such patterns are now accepted, but whenever an iteration of
- such a group matches no characters, matching moves on to the next item
- in the pattern instead of repeatedly matching an empty string. This
- does not prevent backtracking into any of the iterations if a subse-
+ such a group matches no characters, matching moves on to the next item
+ in the pattern instead of repeatedly matching an empty string. This
+ does not prevent backtracking into any of the iterations if a subse-
quent item fails to match.
- By default, quantifiers are "greedy", that is, they match as much as
- possible (up to the maximum number of permitted repetitions), without
- causing the rest of the pattern to fail. The classic example of where
+ By default, quantifiers are "greedy", that is, they match as much as
+ possible (up to the maximum number of permitted repetitions), without
+ causing the rest of the pattern to fail. The classic example of where
this gives problems is in trying to match comments in C programs. These
- appear between /* and */ and within the comment, individual * and /
- characters may appear. An attempt to match C comments by applying the
+ appear between /* and */ and within the comment, individual * and /
+ characters may appear. An attempt to match C comments by applying the
pattern
/\*.*\*/
@@ -8791,17 +8944,17 @@ REPETITION
/* first comment */ not comment /* second comment */
- fails, because it matches the entire string owing to the greediness of
- the .* item. However, if a quantifier is followed by a question mark,
+ fails, because it matches the entire string owing to the greediness of
+ the .* item. However, if a quantifier is followed by a question mark,
it ceases to be greedy, and instead matches the minimum number of times
possible, so the pattern
/\*.*?\*/
- does the right thing with C comments. The meaning of the various quan-
+ does the right thing with C comments. The meaning of the various quan-
tifiers is not otherwise changed, just the preferred number of matches.
- Do not confuse this use of question mark with its use as a quantifier
- in its own right. Because it has two uses, it can sometimes appear
+ Do not confuse this use of question mark with its use as a quantifier
+ in its own right. Because it has two uses, it can sometimes appear
doubled, as in
\d??\d
@@ -8810,56 +8963,56 @@ REPETITION
only way the rest of the pattern matches.
If the PCRE2_UNGREEDY option is set (an option that is not available in
- Perl), the quantifiers are not greedy by default, but individual ones
- can be made greedy by following them with a question mark. In other
+ Perl), the quantifiers are not greedy by default, but individual ones
+ can be made greedy by following them with a question mark. In other
words, it inverts the default behaviour.
- When a parenthesized group is quantified with a minimum repeat count
- that is greater than 1 or with a limited maximum, more memory is re-
+ When a parenthesized group is quantified with a minimum repeat count
+ that is greater than 1 or with a limited maximum, more memory is re-
quired for the compiled pattern, in proportion to the size of the mini-
mum or maximum.
- If a pattern starts with .* or .{0,} and the PCRE2_DOTALL option
- (equivalent to Perl's /s) is set, thus allowing the dot to match new-
- lines, the pattern is implicitly anchored, because whatever follows
- will be tried against every character position in the subject string,
- so there is no point in retrying the overall match at any position af-
- ter the first. PCRE2 normally treats such a pattern as though it were
+ If a pattern starts with .* or .{0,} and the PCRE2_DOTALL option
+ (equivalent to Perl's /s) is set, thus allowing the dot to match new-
+ lines, the pattern is implicitly anchored, because whatever follows
+ will be tried against every character position in the subject string,
+ so there is no point in retrying the overall match at any position af-
+ ter the first. PCRE2 normally treats such a pattern as though it were
preceded by \A.
- In cases where it is known that the subject string contains no new-
- lines, it is worth setting PCRE2_DOTALL in order to obtain this opti-
+ In cases where it is known that the subject string contains no new-
+ lines, it is worth setting PCRE2_DOTALL in order to obtain this opti-
mization, or alternatively, using ^ to indicate anchoring explicitly.
- However, there are some cases where the optimization cannot be used.
- When .* is inside capturing parentheses that are the subject of a
- backreference elsewhere in the pattern, a match at the start may fail
+ However, there are some cases where the optimization cannot be used.
+ When .* is inside capturing parentheses that are the subject of a
+ backreference elsewhere in the pattern, a match at the start may fail
where a later one succeeds. Consider, for example:
(.*)abc\1
- If the subject is "xyz123abc123" the match point is the fourth charac-
+ If the subject is "xyz123abc123" the match point is the fourth charac-
ter. For this reason, such a pattern is not implicitly anchored.
- Another case where implicit anchoring is not applied is when the lead-
- ing .* is inside an atomic group. Once again, a match at the start may
+ Another case where implicit anchoring is not applied is when the lead-
+ ing .* is inside an atomic group. Once again, a match at the start may
fail where a later one succeeds. Consider this pattern:
(?>.*?a)b
- It matches "ab" in the subject "aab". The use of the backtracking con-
- trol verbs (*PRUNE) and (*SKIP) also disable this optimization. To do
- so explicitly, either pass the compile option PCRE2_NO_DOTSTAR_ANCHOR,
+ It matches "ab" in the subject "aab". The use of the backtracking con-
+ trol verbs (*PRUNE) and (*SKIP) also disable this optimization. To do
+ so explicitly, either pass the compile option PCRE2_NO_DOTSTAR_ANCHOR,
or call pcre2_set_optimize() with a PCRE2_DOTSTAR_ANCHOR_OFF directive.
- When a capture group is repeated, the value captured is the substring
+ When a capture group is repeated, the value captured is the substring
that matched the final iteration. For example, after
(tweedle[dume]{3}\s*)+
has matched "tweedledum tweedledee" the value of the captured substring
- is "tweedledee". However, if there are nested capture groups, the cor-
- responding captured values may have been set in previous iterations.
+ is "tweedledee". However, if there are nested capture groups, the cor-
+ responding captured values may have been set in previous iterations.
For example, after
(a|(b))+
@@ -8869,57 +9022,57 @@ REPETITION
ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS
- With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy")
- repetition, failure of what follows normally causes the repeated item
- to be re-evaluated to see if a different number of repeats allows the
- rest of the pattern to match. Sometimes it is useful to prevent this,
- either to change the nature of the match, or to cause it fail earlier
- than it otherwise might, when the author of the pattern knows there is
+ With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy")
+ repetition, failure of what follows normally causes the repeated item
+ to be re-evaluated to see if a different number of repeats allows the
+ rest of the pattern to match. Sometimes it is useful to prevent this,
+ either to change the nature of the match, or to cause it fail earlier
+ than it otherwise might, when the author of the pattern knows there is
no point in carrying on.
- Consider, for example, the pattern \d+foo when applied to the subject
+ Consider, for example, the pattern \d+foo when applied to the subject
line
123456bar
After matching all 6 digits and then failing to match "foo", the normal
- action of the matcher is to try again with only 5 digits matching the
- \d+ item, and then with 4, and so on, before ultimately failing.
- "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides
+ action of the matcher is to try again with only 5 digits matching the
+ \d+ item, and then with 4, and so on, before ultimately failing.
+ "Atomic grouping" (a term taken from Jeffrey Friedl's book) provides
the means for specifying that once a group has matched, it is not to be
re-evaluated in this way.
- If we use atomic grouping for the previous example, the matcher gives
- up immediately on failing to match "foo" the first time. The notation
+ If we use atomic grouping for the previous example, the matcher gives
+ up immediately on failing to match "foo" the first time. The notation
is a kind of special parenthesis, starting with (?> as in this example:
(?>\d+)foo
- Perl 5.28 introduced an experimental alphabetic form starting with (*
+ Perl 5.28 introduced an experimental alphabetic form starting with (*
which may be easier to remember:
(*atomic:\d+)foo
- This kind of parenthesized group "locks up" the part of the pattern it
+ This kind of parenthesized group "locks up" the part of the pattern it
contains once it has matched, and a failure further into the pattern is
- prevented from backtracking into it. Backtracking past it to previous
+ prevented from backtracking into it. Backtracking past it to previous
items, however, works as normal.
An alternative description is that a group of this type matches exactly
- the string of characters that an identical standalone pattern would
+ the string of characters that an identical standalone pattern would
match, if anchored at the current point in the subject string.
- Atomic groups are not capture groups. Simple cases such as the above
- example can be thought of as a maximizing repeat that must swallow
- everything it can. So, while both \d+ and \d+? are prepared to adjust
- the number of digits they match in order to make the rest of the pat-
+ Atomic groups are not capture groups. Simple cases such as the above
+ example can be thought of as a maximizing repeat that must swallow
+ everything it can. So, while both \d+ and \d+? are prepared to adjust
+ the number of digits they match in order to make the rest of the pat-
tern match, (?>\d+) can only match an entire sequence of digits.
- Atomic groups in general can of course contain arbitrarily complicated
+ Atomic groups in general can of course contain arbitrarily complicated
expressions, and can be nested. However, when the contents of an atomic
- group is just a single repeated item, as in the example above, a sim-
- pler notation, called a "possessive quantifier" can be used. This con-
- sists of an additional + character following a quantifier. Using this
+ group is just a single repeated item, as in the example above, a sim-
+ pler notation, called a "possessive quantifier" can be used. This con-
+ sists of an additional + character following a quantifier. Using this
notation, the previous example can be rewritten as
\d++foo
@@ -8929,48 +9082,48 @@ ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS
(abc|xyz){2,3}+
- Possessive quantifiers are always greedy; the setting of the PCRE2_UN-
- GREEDY option is ignored. They are a convenient notation for the sim-
- pler forms of atomic group. However, there is no difference in the
- meaning of a possessive quantifier and the equivalent atomic group,
- though there may be a performance difference; possessive quantifiers
+ Possessive quantifiers are always greedy; the setting of the PCRE2_UN-
+ GREEDY option is ignored. They are a convenient notation for the sim-
+ pler forms of atomic group. However, there is no difference in the
+ meaning of a possessive quantifier and the equivalent atomic group,
+ though there may be a performance difference; possessive quantifiers
should be slightly faster.
- The possessive quantifier syntax is an extension to the Perl 5.8 syn-
- tax. Jeffrey Friedl originated the idea (and the name) in the first
+ The possessive quantifier syntax is an extension to the Perl 5.8 syn-
+ tax. Jeffrey Friedl originated the idea (and the name) in the first
edition of his book. Mike McCloskey liked it, so implemented it when he
- built Sun's Java package, and PCRE1 copied it from there. It found its
+ built Sun's Java package, and PCRE1 copied it from there. It found its
way into Perl at release 5.10.
- PCRE2 has an optimization that automatically "possessifies" certain
- simple pattern constructs. For example, the sequence A+B is treated as
- A++B because there is no point in backtracking into a sequence of A's
- when B must follow. This feature can be disabled by the
- PCRE2_NO_AUTO_POSSESS option, by calling pcre2_set_optimize() with a
- PCRE2_AUTO_POSSESS_OFF directive, or by starting the pattern with
+ PCRE2 has an optimization that automatically "possessifies" certain
+ simple pattern constructs. For example, the sequence A+B is treated as
+ A++B because there is no point in backtracking into a sequence of A's
+ when B must follow. This feature can be disabled by the
+ PCRE2_NO_AUTO_POSSESS option, by calling pcre2_set_optimize() with a
+ PCRE2_AUTO_POSSESS_OFF directive, or by starting the pattern with
(*NO_AUTO_POSSESS).
When a pattern contains an unlimited repeat inside a group that can it-
- self be repeated an unlimited number of times, the use of an atomic
- group is the only way to avoid some failing matches taking a very long
+ self be repeated an unlimited number of times, the use of an atomic
+ group is the only way to avoid some failing matches taking a very long
time indeed. The pattern
(\D+|<\d+>)*[!?]
- matches an unlimited number of substrings that either consist of non-
- digits, or digits enclosed in <>, followed by either ! or ?. When it
+ matches an unlimited number of substrings that either consist of non-
+ digits, or digits enclosed in <>, followed by either ! or ?. When it
matches, it runs quickly. However, if it is applied to
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
- it takes a long time before reporting failure. This is because the
- string can be divided between the internal \D+ repeat and the external
- * repeat in a large number of ways, and all have to be tried. (The ex-
+ it takes a long time before reporting failure. This is because the
+ string can be divided between the internal \D+ repeat and the external
+ * repeat in a large number of ways, and all have to be tried. (The ex-
ample uses [!?] rather than a single character at the end, because both
PCRE2 and Perl have an optimization that allows for fast failure when a
- single character is used. They remember the last single character that
- is required for a match, and fail early if it is not present in the
- string.) If the pattern is changed so that it uses an atomic group,
+ single character is used. They remember the last single character that
+ is required for a match, and fail early if it is not present in the
+ string.) If the pattern is changed so that it uses an atomic group,
like this:
((?>\D+)|<\d+>)*[!?]
@@ -8981,28 +9134,28 @@ ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS
BACKREFERENCES
Outside a character class, a backslash followed by a digit greater than
- 0 (and possibly further digits) is a backreference to a capture group
+ 0 (and possibly further digits) is a backreference to a capture group
earlier (that is, to its left) in the pattern, provided there have been
that many previous capture groups.
- However, if the decimal number following the backslash is less than 8,
- it is always taken as a backreference, and causes an error only if
- there are not that many capture groups in the entire pattern. In other
+ However, if the decimal number following the backslash is less than 8,
+ it is always taken as a backreference, and causes an error only if
+ there are not that many capture groups in the entire pattern. In other
words, the group that is referenced need not be to the left of the ref-
- erence for numbers less than 8. A "forward backreference" of this type
+ erence for numbers less than 8. A "forward backreference" of this type
can make sense when a repetition is involved and the group to the right
has participated in an earlier iteration.
- It is not possible to have a numerical "forward backreference" to a
- group whose number is 8 or more using this syntax because a sequence
- such as \50 is interpreted as a character defined in octal. See the
+ It is not possible to have a numerical "forward backreference" to a
+ group whose number is 8 or more using this syntax because a sequence
+ such as \50 is interpreted as a character defined in octal. See the
subsection entitled "Non-printing characters" above for further details
- of the handling of digits following a backslash. Other forms of back-
- referencing do not suffer from this restriction. In particular, there
+ of the handling of digits following a backslash. Other forms of back-
+ referencing do not suffer from this restriction. In particular, there
is no problem when named capture groups are used (see below).
- Another way of avoiding the ambiguity inherent in the use of digits
- following a backslash is to use the \g escape sequence. This escape
+ Another way of avoiding the ambiguity inherent in the use of digits
+ following a backslash is to use the \g escape sequence. This escape
must be followed by a signed or unsigned number, optionally enclosed in
braces. These examples are all identical:
@@ -9010,54 +9163,54 @@ BACKREFERENCES
(ring), \g1
(ring), \g{1}
- An unsigned number specifies an absolute reference without the ambigu-
+ An unsigned number specifies an absolute reference without the ambigu-
ity that is present in the older syntax. It is also useful when literal
- digits follow the reference. A signed number is a relative reference.
+ digits follow the reference. A signed number is a relative reference.
Consider this example:
(abc(def)ghi)\g{-1}
The sequence \g{-1} is a reference to the capture group whose number is
- one less than the number of the next group to be started, so in this
- example (where the next group would be numbered 3) is it equivalent to
- \2, and \g{-2} would be equivalent to \1. Note that if this construct
- is inside a capture group, that group is included in the count, so in
+ one less than the number of the next group to be started, so in this
+ example (where the next group would be numbered 3) is it equivalent to
+ \2, and \g{-2} would be equivalent to \1. Note that if this construct
+ is inside a capture group, that group is included in the count, so in
this example \g{-2} also refers to group 1:
(A)(\g{-2}B)
- The use of relative references can be helpful in long patterns, and
- also in patterns that are created by joining together fragments that
+ The use of relative references can be helpful in long patterns, and
+ also in patterns that are created by joining together fragments that
contain references within themselves.
- The sequence \g{+1} is a reference to the next capture group that is
- started after this item, and \g{+2} refers to the one after that, and
- so on. This kind of forward reference can be useful in patterns that
+ The sequence \g{+1} is a reference to the next capture group that is
+ started after this item, and \g{+2} refers to the one after that, and
+ so on. This kind of forward reference can be useful in patterns that
repeat. Perl does not support the use of + in this way.
- A backreference matches whatever actually most recently matched the
- capture group in the current subject string, rather than anything at
+ A backreference matches whatever actually most recently matched the
+ capture group in the current subject string, rather than anything at
all that matches the group (see "Groups as subroutines" below for a way
of doing that). So the pattern
(sens|respons)e and \1ibility
- matches "sense and sensibility" and "response and responsibility", but
- not "sense and responsibility". If caseful matching is in force at the
- time of the backreference, the case of letters is relevant. For exam-
+ matches "sense and sensibility" and "response and responsibility", but
+ not "sense and responsibility". If caseful matching is in force at the
+ time of the backreference, the case of letters is relevant. For exam-
ple,
((?i)rah)\s+\1
- matches "rah rah" and "RAH RAH", but not "RAH rah", even though the
+ matches "rah rah" and "RAH RAH", but not "RAH rah", even though the
original capture group is matched caselessly.
- There are several different ways of writing backreferences to named
- capture groups. The .NET syntax is \k{name}, the Python syntax is
- (?=name), and the original Perl syntax is \k or \k'name'. All of
- these are now supported by both Perl and PCRE2. Perl 5.10's unified
- backreference syntax, in which \g can be used for both numeric and
- named references, is also supported by PCRE2. We could rewrite the
+ There are several different ways of writing backreferences to named
+ capture groups. The .NET syntax is \k{name}, the Python syntax is
+ (?=name), and the original Perl syntax is \k or \k'name'. All of
+ these are now supported by both Perl and PCRE2. Perl 5.10's unified
+ backreference syntax, in which \g can be used for both numeric and
+ named references, is also supported by PCRE2. We could rewrite the
above example in any of the following ways:
(?(?i)rah)\s+\k
@@ -9065,120 +9218,120 @@ BACKREFERENCES
(?P(?i)rah)\s+(?P=p1)
(?(?i)rah)\s+\g{p1}
- A capture group that is referenced by name may appear in the pattern
+ A capture group that is referenced by name may appear in the pattern
before or after the reference.
- There may be more than one backreference to the same group. If a group
- has not actually been used in a particular match, backreferences to it
+ There may be more than one backreference to the same group. If a group
+ has not actually been used in a particular match, backreferences to it
always fail by default. For example, the pattern
(a|(bc))\2
- always fails if it starts to match "a" rather than "bc". However, if
+ always fails if it starts to match "a" rather than "bc". However, if
the PCRE2_MATCH_UNSET_BACKREF option is set at compile time, a backref-
erence to an unset value matches an empty string.
- Because there may be many capture groups in a pattern, all digits fol-
- lowing a backslash are taken as part of a potential backreference num-
- ber. If the pattern continues with a digit character, some delimiter
- must be used to terminate the backreference. If the PCRE2_EXTENDED or
- PCRE2_EXTENDED_MORE option is set, this can be white space. Otherwise,
+ Because there may be many capture groups in a pattern, all digits fol-
+ lowing a backslash are taken as part of a potential backreference num-
+ ber. If the pattern continues with a digit character, some delimiter
+ must be used to terminate the backreference. If the PCRE2_EXTENDED or
+ PCRE2_EXTENDED_MORE option is set, this can be white space. Otherwise,
the \g{} syntax or an empty comment (see "Comments" below) can be used.
Recursive backreferences
- A backreference that occurs inside the group to which it refers fails
- when the group is first used, so, for example, (a\1) never matches.
- However, such references can be useful inside repeated groups. For ex-
+ A backreference that occurs inside the group to which it refers fails
+ when the group is first used, so, for example, (a\1) never matches.
+ However, such references can be useful inside repeated groups. For ex-
ample, the pattern
(a|b\1)+
matches any number of "a"s and also "aba", "ababbaa" etc. At each iter-
ation of the group, the backreference matches the character string cor-
- responding to the previous iteration. In order for this to work, the
- pattern must be such that the first iteration does not need to match
- the backreference. This can be done using alternation, as in the exam-
+ responding to the previous iteration. In order for this to work, the
+ pattern must be such that the first iteration does not need to match
+ the backreference. This can be done using alternation, as in the exam-
ple above, or by a quantifier with a minimum of zero.
For versions of PCRE2 less than 10.25, backreferences of this type used
- to cause the group that they reference to be treated as an atomic
- group. This restriction no longer applies, and backtracking into such
+ to cause the group that they reference to be treated as an atomic
+ group. This restriction no longer applies, and backtracking into such
groups can occur as normal.
ASSERTIONS
- An assertion is a test that does not consume any characters. The test
- must succeed for the match to continue. The simple assertions coded as
+ An assertion is a test that does not consume any characters. The test
+ must succeed for the match to continue. The simple assertions coded as
\b, \B, \A, \G, \Z, \z, ^ and $ are described above.
- More complicated assertions are coded as parenthesized groups. If
- matching such a group succeeds, matching continues after it, but with
+ More complicated assertions are coded as parenthesized groups. If
+ matching such a group succeeds, matching continues after it, but with
the matching position in the subject string reset to what it was before
the assertion was processed.
- A special kind of assertion, called a "scan substring" assertion,
- matches a subpattern against a previously captured substring. This is
+ A special kind of assertion, called a "scan substring" assertion,
+ matches a subpattern against a previously captured substring. This is
described in the section entitled "Scan substring assertions" below. It
is a PCRE2 extension, not compatible with Perl.
The other goup-based assertions are of two kinds: those that look ahead
- of the current position in the subject string, and those that look be-
- hind it, and in each case an assertion may be positive (must match for
+ of the current position in the subject string, and those that look be-
+ hind it, and in each case an assertion may be positive (must match for
the assertion to be true) or negative (must not match for the assertion
to be true).
- The Perl-compatible lookaround assertions are atomic. If an assertion
- is true, but there is a subsequent matching failure, there is no back-
- tracking into the assertion. However, there are some cases where non-
- atomic assertions can be useful. PCRE2 has some support for these, de-
+ The Perl-compatible lookaround assertions are atomic. If an assertion
+ is true, but there is a subsequent matching failure, there is no back-
+ tracking into the assertion. However, there are some cases where non-
+ atomic assertions can be useful. PCRE2 has some support for these, de-
scribed in the section entitled "Non-atomic assertions" below, but they
are not Perl-compatible.
- A lookaround assertion may appear as the condition in a conditional
- group (see below). In this case, the result of matching the assertion
+ A lookaround assertion may appear as the condition in a conditional
+ group (see below). In this case, the result of matching the assertion
determines which branch of the condition is followed.
- Assertion groups are not capture groups. If an assertion contains cap-
- ture groups within it, these are counted for the purposes of numbering
- the capture groups in the whole pattern. Within each branch of an as-
- sertion, locally captured substrings may be referenced in the usual
- way. For example, a sequence such as (.)\g{-1} can be used to check
+ Assertion groups are not capture groups. If an assertion contains cap-
+ ture groups within it, these are counted for the purposes of numbering
+ the capture groups in the whole pattern. Within each branch of an as-
+ sertion, locally captured substrings may be referenced in the usual
+ way. For example, a sequence such as (.)\g{-1} can be used to check
that two adjacent characters are the same.
- When a branch within an assertion fails to match, any substrings that
- were captured are discarded (as happens with any pattern branch that
- fails to match). A negative assertion is true only when all its
+ When a branch within an assertion fails to match, any substrings that
+ were captured are discarded (as happens with any pattern branch that
+ fails to match). A negative assertion is true only when all its
branches fail to match; this means that no captured substrings are ever
- retained after a successful negative assertion. When an assertion con-
+ retained after a successful negative assertion. When an assertion con-
tains a matching branch, what happens depends on the type of assertion.
- For a positive assertion, internally captured substrings in the suc-
- cessful branch are retained, and matching continues with the next pat-
- tern item after the assertion. For a negative assertion, a matching
- branch means that the assertion is not true. If such an assertion is
- being used as a condition in a conditional group (see below), captured
- substrings are retained, because matching continues with the "no"
+ For a positive assertion, internally captured substrings in the suc-
+ cessful branch are retained, and matching continues with the next pat-
+ tern item after the assertion. For a negative assertion, a matching
+ branch means that the assertion is not true. If such an assertion is
+ being used as a condition in a conditional group (see below), captured
+ substrings are retained, because matching continues with the "no"
branch of the condition. For other failing negative assertions, control
passes to the previous backtracking point, thus discarding any captured
strings within the assertion.
- Most assertion groups may be repeated; though it makes no sense to as-
+ Most assertion groups may be repeated; though it makes no sense to as-
sert the same thing several times, the side effect of capturing in pos-
itive assertions may occasionally be useful. However, an assertion that
- forms the condition for a conditional group may not be quantified.
- PCRE2 used to restrict the repetition of assertions, but from release
- 10.35 the only restriction is that an unlimited maximum repetition is
- changed to be one more than the minimum. For example, {3,} is treated
+ forms the condition for a conditional group may not be quantified.
+ PCRE2 used to restrict the repetition of assertions, but from release
+ 10.35 the only restriction is that an unlimited maximum repetition is
+ changed to be one more than the minimum. For example, {3,} is treated
as {3,4}.
Alphabetic assertion names
- Traditionally, symbolic sequences such as (?= and (?<= have been used
- to specify lookaround assertions. Perl 5.28 introduced some experimen-
+ Traditionally, symbolic sequences such as (?= and (?<= have been used
+ to specify lookaround assertions. Perl 5.28 introduced some experimen-
tal alphabetic alternatives which might be easier to remember. They all
- start with (* instead of (? and must be written using lower case let-
+ start with (* instead of (? and must be written using lower case let-
ters. PCRE2 supports the following synonyms:
(*positive_lookahead: or (*pla: is the same as (?=
@@ -9186,8 +9339,8 @@ ASSERTIONS
(*positive_lookbehind: or (*plb: is the same as (?<=
(*negative_lookbehind: or (*nlb: is the same as (? .*? \b\1\b ){2}
- For a subject such as "word1 word2 word3 word2 word3 word4" the result
- is "word3". How does it work? At the start, ^(?x) anchors the pattern
+ For a subject such as "word1 word2 word3 word2 word3 word4" the result
+ is "word3". How does it work? At the start, ^(?x) anchors the pattern
and sets the "x" option, which causes white space (introduced for read-
- ability) to be ignored. Inside the assertion, the greedy .* at first
+ ability) to be ignored. Inside the assertion, the greedy .* at first
consumes the entire string, but then has to backtrack until the rest of
- the assertion can match a word, which is captured by group 1. In other
- words, when the assertion first succeeds, it captures the right-most
+ the assertion can match a word, which is captured by group 1. In other
+ words, when the assertion first succeeds, it captures the right-most
word in the string.
- The current matching point is then reset to the start of the subject,
- and the rest of the pattern match checks for two occurrences of the
- captured word, using an ungreedy .*? to scan from the left. If this
- succeeds, we are done, but if the last word in the string does not oc-
- cur twice, this part of the pattern fails. If a traditional atomic
- lookahead (?= or (*pla: had been used, the assertion could not be re-
+ The current matching point is then reset to the start of the subject,
+ and the rest of the pattern match checks for two occurrences of the
+ captured word, using an ungreedy .*? to scan from the left. If this
+ succeeds, we are done, but if the last word in the string does not oc-
+ cur twice, this part of the pattern fails. If a traditional atomic
+ lookahead (?= or (*pla: had been used, the assertion could not be re-
entered, and the whole match would fail. The pattern would succeed only
if the very last word in the subject was found twice.
- Using a non-atomic lookahead, however, means that when the last word
- does not occur twice in the string, the lookahead can backtrack and
- find the second-last word, and so on, until either the match succeeds,
+ Using a non-atomic lookahead, however, means that when the last word
+ does not occur twice in the string, the lookahead can backtrack and
+ find the second-last word, and so on, until either the match succeeds,
or all words have been tested.
Two conditions must be met for a non-atomic assertion to be useful: the
- contents of one or more capturing groups must change after a backtrack
- into the assertion, and there must be a backreference to a changed
- group later in the pattern. If this is not the case, the rest of the
- pattern match fails exactly as before because nothing has changed, so
+ contents of one or more capturing groups must change after a backtrack
+ into the assertion, and there must be a backreference to a changed
+ group later in the pattern. If this is not the case, the rest of the
+ pattern match fails exactly as before because nothing has changed, so
using a non-atomic assertion just wastes resources.
- There is one exception to backtracking into a non-atomic assertion. If
- an (*ACCEPT) control verb is triggered, the assertion succeeds atomi-
- cally. That is, a subsequent match failure cannot backtrack into the
+ There is one exception to backtracking into a non-atomic assertion. If
+ an (*ACCEPT) control verb is triggered, the assertion succeeds atomi-
+ cally. That is, a subsequent match failure cannot backtrack into the
assertion.
- Non-atomic assertions are not supported by the alternative matching
+ Non-atomic assertions are not supported by the alternative matching
function pcre2_dfa_match(). They are supported by JIT, but only if they
do not contain any control verbs such as (*ACCEPT). (This may change in
future). Note that assertions that appear as conditions for conditional
@@ -9396,15 +9549,15 @@ NON-ATOMIC ASSERTIONS
SCAN SUBSTRING ASSERTIONS
- A special kind of assertion, not compatible with Perl, makes it possi-
+ A special kind of assertion, not compatible with Perl, makes it possi-
ble to check the contents of a captured substring by matching it with a
- subpattern. Because this involves capturing, this feature is not sup-
+ subpattern. Because this involves capturing, this feature is not sup-
ported by pcre2_dfa_match().
- A scan substring assertion starts with the sequence (*scan_substring:
+ A scan substring assertion starts with the sequence (*scan_substring:
or (*scs: which is followed by a list of substring numbers (absolute or
- relative) and/or substring names enclosed in single quotes or angle
- brackets, all within parentheses. The rest of the item is the subpat-
+ relative) and/or substring names enclosed in single quotes or angle
+ brackets, all within parentheses. The rest of the item is the subpat-
tern that is applied to the substring, as shown in these examples:
(*scan_substring:(1)...)
@@ -9412,86 +9565,86 @@ SCAN SUBSTRING ASSERTIONS
(*scs:('AB')...)
(*scs:(1,'AB',-2)...)
- The list of groups is checked in the order they are given, and it is
+ The list of groups is checked in the order they are given, and it is
the contents of the first one that is found to be set that are scanned.
- When PCRE2_DUPNAMES is set and there are ambiguous group names, all
- groups with the same name are checked in numerical order. A scan sub-
- string assertion fails if none of the groups it references have been
+ When PCRE2_DUPNAMES is set and there are ambiguous group names, all
+ groups with the same name are checked in numerical order. A scan sub-
+ string assertion fails if none of the groups it references have been
set.
The pattern match on the substring is always anchored, that is, it must
- match from the start of the substring. There is no "bumpalong" if it
- does not match at the start. The end of the subject is temporarily re-
- set to be the end of the substring, so \Z, \z, and $ will match there.
- However, the start of the subject is not reset. This means that ^
+ match from the start of the substring. There is no "bumpalong" if it
+ does not match at the start. The end of the subject is temporarily re-
+ set to be the end of the substring, so \Z, \z, and $ will match there.
+ However, the start of the subject is not reset. This means that ^
matches only if the substring is actually at the start of the main sub-
- ject, but it also means that lookbehind assertions into what precedes
+ ject, but it also means that lookbehind assertions into what precedes
the substring are possible.
- Here is a very simple example: find a word that contains the rare (in
+ Here is a very simple example: find a word that contains the rare (in
English) sequence of letters "rh" not at the start:
\b(\w++)(*scs:(1).+rh)
- The first group captures a word which is then scanned by the second
- group. This example does not actually need this heavyweight feature;
+ The first group captures a word which is then scanned by the second
+ group. This example does not actually need this heavyweight feature;
the same match can be achieved with:
\b\w+?rh\w*\b
- When things are more complicated, however, scanning a captured sub-
- string can be a useful way to describe the required match. For exmple,
- there is a rather complicated pattern in the PCRE2 test data that
+ When things are more complicated, however, scanning a captured sub-
+ string can be a useful way to describe the required match. For exmple,
+ there is a rather complicated pattern in the PCRE2 test data that
checks an entire subject string for a palindrome, that is, the sequence
- of letters is the same in both directions. Suppose you want to search
+ of letters is the same in both directions. Suppose you want to search
for individual words of two or more characters such as "level" that are
palindromes:
(\b\w{2,}+\b)(*scs:(1)...palindrome-matching-pattern...)
Within a substring scanning subpattern, references to other groups work
- as normal. Capturing groups may appear, and will retain their values
+ as normal. Capturing groups may appear, and will retain their values
during ongoing matching if the assertion succeeds.
SCRIPT RUNS
- In concept, a script run is a sequence of characters that are all from
- the same Unicode script such as Latin or Greek. However, because some
- scripts are commonly used together, and because some diacritical and
- other marks are used with multiple scripts, it is not that simple.
+ In concept, a script run is a sequence of characters that are all from
+ the same Unicode script such as Latin or Greek. However, because some
+ scripts are commonly used together, and because some diacritical and
+ other marks are used with multiple scripts, it is not that simple.
There is a full description of the rules that PCRE2 uses in the section
entitled "Script Runs" in the pcre2unicode documentation.
- If part of a pattern is enclosed between (*script_run: or (*sr: and a
- closing parenthesis, it fails if the sequence of characters that it
- matches are not a script run. After a failure, normal backtracking oc-
- curs. Script runs can be used to detect spoofing attacks using charac-
- ters that look the same, but are from different scripts. The string
- "paypal.com" is an infamous example, where the letters could be a mix-
+ If part of a pattern is enclosed between (*script_run: or (*sr: and a
+ closing parenthesis, it fails if the sequence of characters that it
+ matches are not a script run. After a failure, normal backtracking oc-
+ curs. Script runs can be used to detect spoofing attacks using charac-
+ ters that look the same, but are from different scripts. The string
+ "paypal.com" is an infamous example, where the letters could be a mix-
ture of Latin and Cyrillic. This pattern ensures that the matched char-
acters in a sequence of non-spaces that follow white space are a script
run:
\s+(*sr:\S+)
- To be sure that they are all from the Latin script (for example), a
+ To be sure that they are all from the Latin script (for example), a
lookahead can be used:
\s+(?=\p{Latin})(*sr:\S+)
This works as long as the first character is expected to be a character
- in that script, and not (for example) punctuation, which is allowed
- with any script. If this is not the case, a more creative lookahead is
- needed. For example, if digits, underscore, and dots are permitted at
+ in that script, and not (for example) punctuation, which is allowed
+ with any script. If this is not the case, a more creative lookahead is
+ needed. For example, if digits, underscore, and dots are permitted at
the start:
\s+(?=[0-9_.]*\p{Latin})(*sr:\S+)
- In many cases, backtracking into a script run pattern fragment is not
- desirable. The script run can employ an atomic group to prevent this.
- Because this is a common requirement, a shorthand notation is provided
+ In many cases, backtracking into a script run pattern fragment is not
+ desirable. The script run can employ an atomic group to prevent this.
+ Because this is a common requirement, a shorthand notation is provided
by (*atomic_script_run: or (*asr:
(*asr:...) is the same as (*sr:(?>...))
@@ -9499,13 +9652,13 @@ SCRIPT RUNS
Note that the atomic group is inside the script run. Putting it outside
would not prevent backtracking into the script run pattern.
- Support for script runs is not available if PCRE2 is compiled without
+ Support for script runs is not available if PCRE2 is compiled without
Unicode support. A compile-time error is given if any of the above con-
- structs is encountered. Script runs are not supported by the alternate
- matching function, pcre2_dfa_match() because they use the same mecha-
+ structs is encountered. Script runs are not supported by the alternate
+ matching function, pcre2_dfa_match() because they use the same mecha-
nism as capturing parentheses.
- Warning: The (*ACCEPT) control verb (see below) should not be used
+ Warning: The (*ACCEPT) control verb (see below) should not be used
within a script run group, because it causes an immediate exit from the
group, bypassing the script run checking.
@@ -9514,117 +9667,117 @@ CONDITIONAL GROUPS
It is possible to cause the matching process to obey a pattern fragment
conditionally or to choose between two alternative fragments, depending
- on the result of an assertion, or whether a specific capture group has
+ on the result of an assertion, or whether a specific capture group has
already been matched. The two possible forms of conditional group are:
(?(condition)yes-pattern)
(?(condition)yes-pattern|no-pattern)
- If the condition is satisfied, the yes-pattern is used; otherwise the
- no-pattern (if present) is used. An absent no-pattern is equivalent to
- an empty string (it always matches). If there are more than two alter-
- natives in the group, a compile-time error occurs. Each of the two al-
+ If the condition is satisfied, the yes-pattern is used; otherwise the
+ no-pattern (if present) is used. An absent no-pattern is equivalent to
+ an empty string (it always matches). If there are more than two alter-
+ natives in the group, a compile-time error occurs. Each of the two al-
ternatives may itself contain nested groups of any form, including con-
- ditional groups; the restriction to two alternatives applies only at
- the level of the condition itself. This pattern fragment is an example
+ ditional groups; the restriction to two alternatives applies only at
+ the level of the condition itself. This pattern fragment is an example
where the alternatives are complex:
(?(1) (A|B|C) | (D | (?(2)E|F) | E) )
There are five kinds of condition: references to capture groups, refer-
- ences to recursion, two pseudo-conditions called DEFINE and VERSION,
+ ences to recursion, two pseudo-conditions called DEFINE and VERSION,
and assertions.
Checking for a used capture group by number
- If the text between the parentheses consists of a sequence of digits,
- the condition is true if a capture group of that number has previously
- matched. If there is more than one capture group with the same number
- (see the earlier section about duplicate group numbers), the condition
- is true if any of them have matched. An alternative notation, which is
+ If the text between the parentheses consists of a sequence of digits,
+ the condition is true if a capture group of that number has previously
+ matched. If there is more than one capture group with the same number
+ (see the earlier section about duplicate group numbers), the condition
+ is true if any of them have matched. An alternative notation, which is
a PCRE2 extension, not supported by Perl, is to precede the digits with
a plus or minus sign. In this case, the group number is relative rather
- than absolute. The most recently opened capture group (which could be
- enclosing this condition) can be referenced by (?(-1), the next most
+ than absolute. The most recently opened capture group (which could be
+ enclosing this condition) can be referenced by (?(-1), the next most
recent by (?(-2), and so on. Inside loops it can also make sense to re-
- fer to subsequent groups. The next capture group to be opened can be
- referenced as (?(+1), and so on. The value zero in any of these forms
+ fer to subsequent groups. The next capture group to be opened can be
+ referenced as (?(+1), and so on. The value zero in any of these forms
is not used; it provokes a compile-time error.
- Consider the following pattern, which contains non-significant white
- space to make it more readable (assume the PCRE2_EXTENDED option) and
+ Consider the following pattern, which contains non-significant white
+ space to make it more readable (assume the PCRE2_EXTENDED option) and
to divide it into three parts for ease of discussion:
( \( )? [^()]+ (?(1) \) )
- The first part matches an optional opening parenthesis, and if that
+ The first part matches an optional opening parenthesis, and if that
character is present, sets it as the first captured substring. The sec-
- ond part matches one or more characters that are not parentheses. The
- third part is a conditional group that tests whether or not the first
- capture group matched. If it did, that is, if subject started with an
- opening parenthesis, the condition is true, and so the yes-pattern is
- executed and a closing parenthesis is required. Otherwise, since no-
+ ond part matches one or more characters that are not parentheses. The
+ third part is a conditional group that tests whether or not the first
+ capture group matched. If it did, that is, if subject started with an
+ opening parenthesis, the condition is true, and so the yes-pattern is
+ executed and a closing parenthesis is required. Otherwise, since no-
pattern is not present, the conditional group matches nothing. In other
- words, this pattern matches a sequence of non-parentheses, optionally
+ words, this pattern matches a sequence of non-parentheses, optionally
enclosed in parentheses.
- If you were embedding this pattern in a larger one, you could use a
+ If you were embedding this pattern in a larger one, you could use a
relative reference:
...other stuff... ( \( )? [^()]+ (?(-1) \) ) ...
- This makes the fragment independent of the parentheses in the larger
+ This makes the fragment independent of the parentheses in the larger
pattern.
Checking for a used capture group by name
- Perl uses the syntax (?()...) or (?('name')...) to test for a
- used capture group by name. For compatibility with earlier versions of
- PCRE1, which had this facility before Perl, the syntax (?(name)...) is
- also recognized. Note, however, that undelimited names consisting of
- the letter R followed by digits are ambiguous (see the following sec-
+ Perl uses the syntax (?()...) or (?('name')...) to test for a
+ used capture group by name. For compatibility with earlier versions of
+ PCRE1, which had this facility before Perl, the syntax (?(name)...) is
+ also recognized. Note, however, that undelimited names consisting of
+ the letter R followed by digits are ambiguous (see the following sec-
tion). Rewriting the above example to use a named group gives this:
(? \( )? [^()]+ (?() \) )
- If the name used in a condition of this kind is a duplicate, the test
- is applied to all groups of the same name, and is true if any one of
+ If the name used in a condition of this kind is a duplicate, the test
+ is applied to all groups of the same name, and is true if any one of
them has matched.
Checking for pattern recursion
- "Recursion" in this sense refers to any subroutine-like call from one
- part of the pattern to another, whether or not it is actually recur-
- sive. See the sections entitled "Recursive patterns" and "Groups as
+ "Recursion" in this sense refers to any subroutine-like call from one
+ part of the pattern to another, whether or not it is actually recur-
+ sive. See the sections entitled "Recursive patterns" and "Groups as
subroutines" below for details of recursion and subroutine calls.
- If a condition is the string (R), and there is no capture group with
- the name R, the condition is true if matching is currently in a recur-
- sion or subroutine call to the whole pattern or any capture group. If
- digits follow the letter R, and there is no group with that name, the
- condition is true if the most recent call is into a group with the
- given number, which must exist somewhere in the overall pattern. This
+ If a condition is the string (R), and there is no capture group with
+ the name R, the condition is true if matching is currently in a recur-
+ sion or subroutine call to the whole pattern or any capture group. If
+ digits follow the letter R, and there is no group with that name, the
+ condition is true if the most recent call is into a group with the
+ given number, which must exist somewhere in the overall pattern. This
is a contrived example that is equivalent to a+b:
((?(R1)a+|(?1)b))
- However, in both cases, if there is a capture group with a matching
- name, the condition tests for its being set, as described in the sec-
- tion above, instead of testing for recursion. For example, creating a
- group with the name R1 by adding (?) to the above pattern com-
+ However, in both cases, if there is a capture group with a matching
+ name, the condition tests for its being set, as described in the sec-
+ tion above, instead of testing for recursion. For example, creating a
+ group with the name R1 by adding (?) to the above pattern com-
pletely changes its meaning.
If a name preceded by ampersand follows the letter R, for example:
(?(R&name)...)
- the condition is true if the most recent recursion is into a group of
+ the condition is true if the most recent recursion is into a group of
that name (which must exist within the pattern).
This condition does not check the entire recursion stack. It tests only
- the current level. If the name used in a condition of this kind is a
- duplicate, the test is applied to all groups of the same name, and is
+ the current level. If the name used in a condition of this kind is a
+ duplicate, the test is applied to all groups of the same name, and is
true if any one of them is the most recent recursion.
At "top level", all these recursion test conditions are false.
@@ -9632,112 +9785,112 @@ CONDITIONAL GROUPS
Defining capture groups for use by reference only
If the condition is the string (DEFINE), the condition is always false,
- even if there is a group with the name DEFINE. In this case, there may
+ even if there is a group with the name DEFINE. In this case, there may
be only one alternative in the rest of the conditional group. It is al-
- ways skipped if control reaches this point in the pattern; the idea of
- DEFINE is that it can be used to define subroutines that can be refer-
- enced from elsewhere. (The use of subroutines is described below.) For
- example, a pattern to match an IPv4 address such as "192.168.23.245"
+ ways skipped if control reaches this point in the pattern; the idea of
+ DEFINE is that it can be used to define subroutines that can be refer-
+ enced from elsewhere. (The use of subroutines is described below.) For
+ example, a pattern to match an IPv4 address such as "192.168.23.245"
could be written like this (ignore white space and line breaks):
(?(DEFINE) (? 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) )
\b (?&byte) (\.(?&byte)){3} \b
- The first part of the pattern is a DEFINE group inside which another
- group named "byte" is defined. This matches an individual component of
- an IPv4 address (a number less than 256). When matching takes place,
- this part of the pattern is skipped because DEFINE acts like a false
- condition. The rest of the pattern uses references to the named group
- to match the four dot-separated components of an IPv4 address, insist-
+ The first part of the pattern is a DEFINE group inside which another
+ group named "byte" is defined. This matches an individual component of
+ an IPv4 address (a number less than 256). When matching takes place,
+ this part of the pattern is skipped because DEFINE acts like a false
+ condition. The rest of the pattern uses references to the named group
+ to match the four dot-separated components of an IPv4 address, insist-
ing on a word boundary at each end.
Checking the PCRE2 version
- Programs that link with a PCRE2 library can check the version by call-
- ing pcre2_config() with appropriate arguments. Users of applications
- that do not have access to the underlying code cannot do this. A spe-
- cial "condition" called VERSION exists to allow such users to discover
+ Programs that link with a PCRE2 library can check the version by call-
+ ing pcre2_config() with appropriate arguments. Users of applications
+ that do not have access to the underlying code cannot do this. A spe-
+ cial "condition" called VERSION exists to allow such users to discover
which version of PCRE2 they are dealing with by using this condition to
- match a string such as "yesno". VERSION must be followed either by "="
+ match a string such as "yesno". VERSION must be followed either by "="
or ">=" and a version number. For example:
(?(VERSION>=10.4)yes|no)
- This pattern matches "yes" if the PCRE2 version is greater or equal to
- 10.4, or "no" otherwise. The fractional part of the version number may
- not contain more than two digits.
+ This pattern matches "yes" if the PCRE2 version is greater or equal to
+ 10.4, or "no" otherwise. The fractional part of the version number
+ could be ommited.
Assertion conditions
- If the condition is not in any of the above formats, it must be a
- parenthesized assertion. This may be a positive or negative lookahead
- or lookbehind assertion. However, it must be a traditional atomic as-
+ If the condition is not in any of the above formats, it must be a
+ parenthesized assertion. This may be a positive or negative lookahead
+ or lookbehind assertion. However, it must be a traditional atomic as-
sertion, not one of the non-atomic assertions.
- Consider this pattern, again containing non-significant white space,
+ Consider this pattern, again containing non-significant white space,
and with the two alternatives on the second line:
(?(?=[^a-z]*[a-z])
\d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} )
- The condition is a positive lookahead assertion that matches an op-
+ The condition is a positive lookahead assertion that matches an op-
tional sequence of non-letters followed by a letter. In other words, it
tests for the presence of at least one letter in the subject. If a let-
- ter is found, the subject is matched against the first alternative;
- otherwise it is matched against the second. This pattern matches
- strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are
+ ter is found, the subject is matched against the first alternative;
+ otherwise it is matched against the second. This pattern matches
+ strings in one of the two forms dd-aaa-dd or dd-dd-dd, where aaa are
letters and dd are digits.
When an assertion that is a condition contains capture groups, any cap-
- turing that occurs in a matching branch is retained afterwards, for
- both positive and negative assertions, because matching always contin-
- ues after the assertion, whether it succeeds or fails. (Compare non-
- conditional assertions, for which captures are retained only for posi-
+ turing that occurs in a matching branch is retained afterwards, for
+ both positive and negative assertions, because matching always contin-
+ ues after the assertion, whether it succeeds or fails. (Compare non-
+ conditional assertions, for which captures are retained only for posi-
tive assertions that succeed.)
COMMENTS
There are two ways of including comments in patterns that are processed
- by PCRE2. In both cases, the start of the comment must not be in a
- character class, nor in the middle of any other sequence of related
- characters such as (?: or a group name or number or a Unicode property
+ by PCRE2. In both cases, the start of the comment must not be in a
+ character class, nor in the middle of any other sequence of related
+ characters such as (?: or a group name or number or a Unicode property
name. The characters that make up a comment play no part in the pattern
matching.
- The sequence (?# marks the start of a comment that continues up to the
- next closing parenthesis. Nested parentheses are not permitted. If the
- PCRE2_EXTENDED or PCRE2_EXTENDED_MORE option is set, an unescaped #
- character also introduces a comment, which in this case continues to
- immediately after the next newline character or character sequence in
+ The sequence (?# marks the start of a comment that continues up to the
+ next closing parenthesis. Nested parentheses are not permitted. If the
+ PCRE2_EXTENDED or PCRE2_EXTENDED_MORE option is set, an unescaped #
+ character also introduces a comment, which in this case continues to
+ immediately after the next newline character or character sequence in
the pattern. Which characters are interpreted as newlines is controlled
- by an option passed to the compiling function or by a special sequence
+ by an option passed to the compiling function or by a special sequence
at the start of the pattern, as described in the section entitled "New-
line conventions" above. Note that the end of this type of comment is a
- literal newline sequence in the pattern; escape sequences that happen
+ literal newline sequence in the pattern; escape sequences that happen
to represent a newline do not count. For example, consider this pattern
- when PCRE2_EXTENDED is set, and the default newline convention (a sin-
+ when PCRE2_EXTENDED is set, and the default newline convention (a sin-
gle linefeed character) is in force:
abc #comment \n still comment
- On encountering the # character, pcre2_compile() skips along, looking
- for a newline in the pattern. The sequence \n is still literal at this
- stage, so it does not terminate the comment. Only an actual character
+ On encountering the # character, pcre2_compile() skips along, looking
+ for a newline in the pattern. The sequence \n is still literal at this
+ stage, so it does not terminate the comment. Only an actual character
with the code value 0x0a (the default newline) does so.
RECURSIVE PATTERNS
- Consider the problem of matching a string in parentheses, allowing for
- unlimited nested parentheses. Without the use of recursion, the best
- that can be done is to use a pattern that matches up to some fixed
- depth of nesting. It is not possible to handle an arbitrary nesting
+ Consider the problem of matching a string in parentheses, allowing for
+ unlimited nested parentheses. Without the use of recursion, the best
+ that can be done is to use a pattern that matches up to some fixed
+ depth of nesting. It is not possible to handle an arbitrary nesting
depth.
For some time, Perl has provided a facility that allows regular expres-
- sions to recurse (amongst other things). It does this by interpolating
- Perl code in the expression at run time, and the code can refer to the
+ sions to recurse (amongst other things). It does this by interpolating
+ Perl code in the expression at run time, and the code can refer to the
expression itself. A Perl pattern using code interpolation to solve the
parentheses problem can be created like this:
@@ -9746,67 +9899,67 @@ RECURSIVE PATTERNS
The (?p{...}) item interpolates Perl code at run time, and in this case
refers recursively to the pattern in which it appears.
- Obviously, PCRE2 cannot support the interpolation of Perl code. In-
- stead, it supports special syntax for recursion of the entire pattern,
+ Obviously, PCRE2 cannot support the interpolation of Perl code. In-
+ stead, it supports special syntax for recursion of the entire pattern,
and also for individual capture group recursion. After its introduction
in PCRE1 and Python, this kind of recursion was subsequently introduced
into Perl at release 5.10.
- A special item that consists of (? followed by a number greater than
- zero and a closing parenthesis is a recursive subroutine call of the
- capture group of the given number, provided that it occurs inside that
- group. (If not, it is a non-recursive subroutine call, which is de-
+ A special item that consists of (? followed by a number greater than
+ zero and a closing parenthesis is a recursive subroutine call of the
+ capture group of the given number, provided that it occurs inside that
+ group. (If not, it is a non-recursive subroutine call, which is de-
scribed in the next section.) The special item (?R) or (?0) is a recur-
sive call of the entire regular expression.
- This PCRE2 pattern solves the nested parentheses problem (assume the
+ This PCRE2 pattern solves the nested parentheses problem (assume the
PCRE2_EXTENDED option is set so that white space is ignored):
\( ( [^()]++ | (?R) )* \)
- First it matches an opening parenthesis. Then it matches any number of
- substrings which can either be a sequence of non-parentheses, or a re-
+ First it matches an opening parenthesis. Then it matches any number of
+ substrings which can either be a sequence of non-parentheses, or a re-
cursive match of the pattern itself (that is, a correctly parenthesized
- substring). Finally there is a closing parenthesis. Note the use of a
- possessive quantifier to avoid backtracking into sequences of non-
+ substring). Finally there is a closing parenthesis. Note the use of a
+ possessive quantifier to avoid backtracking into sequences of non-
parentheses.
- If this were part of a larger pattern, you would not want to recurse
+ If this were part of a larger pattern, you would not want to recurse
the entire pattern, so instead you could use this:
( \( ( [^()]++ | (?1) )* \) )
- We have put the pattern into parentheses, and caused the recursion to
+ We have put the pattern into parentheses, and caused the recursion to
refer to them instead of the whole pattern.
- In a larger pattern, keeping track of parenthesis numbers can be
- tricky. This is made easier by the use of relative references. Instead
+ In a larger pattern, keeping track of parenthesis numbers can be
+ tricky. This is made easier by the use of relative references. Instead
of (?1) in the pattern above you can write (?-2) to refer to the second
- most recently opened parentheses preceding the recursion. In other
- words, a negative number counts capturing parentheses leftwards from
+ most recently opened parentheses preceding the recursion. In other
+ words, a negative number counts capturing parentheses leftwards from
the point at which it is encountered.
- Be aware however, that if duplicate capture group numbers are in use,
- relative references refer to the earliest group with the appropriate
+ Be aware however, that if duplicate capture group numbers are in use,
+ relative references refer to the earliest group with the appropriate
number. Consider, for example:
(?|(a)|(b)) (c) (?-2)
The first two capture groups (a) and (b) are both numbered 1, and group
- (c) is number 2. When the reference (?-2) is encountered, the second
- most recently opened parentheses has the number 1, but it is the first
+ (c) is number 2. When the reference (?-2) is encountered, the second
+ most recently opened parentheses has the number 1, but it is the first
such group (the (a) group) to which the recursion refers. This would be
- the same if an absolute reference (?1) was used. In other words, rela-
+ the same if an absolute reference (?1) was used. In other words, rela-
tive references are just a shorthand for computing a group number.
- It is also possible to refer to subsequent capture groups, by writing
- references such as (?+2). However, these cannot be recursive because
- the reference is not inside the parentheses that are referenced. They
- are always non-recursive subroutine calls, as described in the next
+ It is also possible to refer to subsequent capture groups, by writing
+ references such as (?+2). However, these cannot be recursive because
+ the reference is not inside the parentheses that are referenced. They
+ are always non-recursive subroutine calls, as described in the next
section.
- An alternative approach is to use named parentheses. The Perl syntax
- for this is (?&name); PCRE1's earlier syntax (?P>name) is also sup-
+ An alternative approach is to use named parentheses. The Perl syntax
+ for this is (?&name); PCRE1's earlier syntax (?P>name) is also sup-
ported. We could rewrite the above example as follows:
(? \( ( [^()]++ | (?&pn) )* \) )
@@ -9815,57 +9968,57 @@ RECURSIVE PATTERNS
used.
The example pattern that we have been looking at contains nested unlim-
- ited repeats, and so the use of a possessive quantifier for matching
- strings of non-parentheses is important when applying the pattern to
+ ited repeats, and so the use of a possessive quantifier for matching
+ strings of non-parentheses is important when applying the pattern to
strings that do not match. For example, when this pattern is applied to
(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()
- it yields "no match" quickly. However, if a possessive quantifier is
- not used, the match runs for a very long time indeed because there are
- so many different ways the + and * repeats can carve up the subject,
+ it yields "no match" quickly. However, if a possessive quantifier is
+ not used, the match runs for a very long time indeed because there are
+ so many different ways the + and * repeats can carve up the subject,
and all have to be tested before failure can be reported.
- At the end of a match, the values of capturing parentheses are those
- from the outermost level. If you want to obtain intermediate values, a
+ At the end of a match, the values of capturing parentheses are those
+ from the outermost level. If you want to obtain intermediate values, a
callout function can be used (see below and the pcre2callout documenta-
tion). If the pattern above is matched against
(ab(cd)ef)
- the value for the inner capturing parentheses (numbered 2) is "ef",
- which is the last value taken on at the top level. If a capture group
- is not matched at the top level, its final captured value is unset,
- even if it was (temporarily) set at a deeper level during the matching
+ the value for the inner capturing parentheses (numbered 2) is "ef",
+ which is the last value taken on at the top level. If a capture group
+ is not matched at the top level, its final captured value is unset,
+ even if it was (temporarily) set at a deeper level during the matching
process.
- Do not confuse the (?R) item with the condition (R), which tests for
- recursion. Consider this pattern, which matches text in angle brack-
- ets, allowing for arbitrary nesting. Only digits are allowed in nested
- brackets (that is, when recursing), whereas any characters are permit-
+ Do not confuse the (?R) item with the condition (R), which tests for
+ recursion. Consider this pattern, which matches text in angle brack-
+ ets, allowing for arbitrary nesting. Only digits are allowed in nested
+ brackets (that is, when recursing), whereas any characters are permit-
ted at the outer level.
< (?: (?(R) \d++ | [^<>]*+) | (?R)) * >
- In this pattern, (?(R) is the start of a conditional group, with two
- different alternatives for the recursive and non-recursive cases. The
+ In this pattern, (?(R) is the start of a conditional group, with two
+ different alternatives for the recursive and non-recursive cases. The
(?R) item is the actual recursive call.
Differences in recursion processing between PCRE2 and Perl
Some former differences between PCRE2 and Perl no longer exist.
- Before release 10.30, recursion processing in PCRE2 differed from Perl
- in that a recursive subroutine call was always treated as an atomic
- group. That is, once it had matched some of the subject string, it was
- never re-entered, even if it contained untried alternatives and there
- was a subsequent matching failure. (Historical note: PCRE implemented
+ Before release 10.30, recursion processing in PCRE2 differed from Perl
+ in that a recursive subroutine call was always treated as an atomic
+ group. That is, once it had matched some of the subject string, it was
+ never re-entered, even if it contained untried alternatives and there
+ was a subsequent matching failure. (Historical note: PCRE implemented
recursion before Perl did.)
- Starting with release 10.30, recursive subroutine calls are no longer
+ Starting with release 10.30, recursive subroutine calls are no longer
treated as atomic. That is, they can be re-entered to try unused alter-
- natives if there is a matching failure later in the pattern. This is
- now compatible with the way Perl works. If you want a subroutine call
+ natives if there is a matching failure later in the pattern. This is
+ now compatible with the way Perl works. If you want a subroutine call
to be atomic, you must explicitly enclose it in an atomic group.
Supporting backtracking into recursions simplifies certain types of re-
@@ -9873,47 +10026,46 @@ RECURSIVE PATTERNS
^((.)(?1)\2|.?)$
- The second branch in the group matches a single central character in
- the palindrome when there are an odd number of characters, or nothing
- when there are an even number of characters, but in order to work it
- has to be able to try the second case when the rest of the pattern
+ The second branch in the group matches a single central character in
+ the palindrome when there are an odd number of characters, or nothing
+ when there are an even number of characters, but in order to work it
+ has to be able to try the second case when the rest of the pattern
match fails. If you want to match typical palindromic phrases, the pat-
- tern has to ignore all non-word characters, which can be done like
+ tern has to ignore all non-word characters, which can be done like
this:
^\W*+((.)\W*+(?1)\W*+\2|\W*+.?)\W*+$
- If run with the PCRE2_CASELESS option, this pattern matches phrases
- such as "A man, a plan, a canal: Panama!". Note the use of the posses-
- sive quantifier *+ to avoid backtracking into sequences of non-word
+ If run with the PCRE2_CASELESS option, this pattern matches phrases
+ such as "A man, a plan, a canal: Panama!". Note the use of the posses-
+ sive quantifier *+ to avoid backtracking into sequences of non-word
characters. Without this, PCRE2 takes a great deal longer (ten times or
- more) to match typical phrases, and Perl takes so long that you think
+ more) to match typical phrases, and Perl takes so long that you think
it has gone into a loop.
- Another way in which PCRE2 and Perl used to differ in their recursion
- processing is in the handling of captured values. Formerly in Perl,
- when a group was called recursively or as a subroutine (see the next
+ Another way in which PCRE2 and Perl used to differ in their recursion
+ processing is in the handling of captured values. Formerly in Perl,
+ when a group was called recursively or as a subroutine (see the next
section), it had no access to any values that were captured outside the
- recursion, whereas in PCRE2 these values can be referenced. Consider
+ recursion, whereas in PCRE2 these values can be referenced. Consider
this pattern:
^(.)(\1|a(?2))
- This pattern matches "bab". The first capturing parentheses match "b",
+ This pattern matches "bab". The first capturing parentheses match "b",
then in the second group, when the backreference \1 fails to match "b",
the second alternative matches "a" and then recurses. In the recursion,
- \1 does now match "b" and so the whole match succeeds. This match used
+ \1 does now match "b" and so the whole match succeeds. This match used
to fail in Perl, but in later versions (I tried 5.024) it now works.
+ Groups as subroutines
-GROUPS AS SUBROUTINES
-
- If the syntax for a recursive group call (either by number or by name)
- is used outside the parentheses to which it refers, it operates a bit
- like a subroutine in a programming language. More accurately, PCRE2
+ If the syntax for a recursive group call (either by number or by name)
+ is used outside the parentheses to which it refers, it operates a bit
+ like a subroutine in a programming language. More accurately, PCRE2
treats the referenced group as an independent subpattern which it tries
- to match at the current matching position. The called group may be de-
- fined before or after the reference. A numbered reference can be ab-
+ to match at the current matching position. The called group may be de-
+ fined before or after the reference. A numbered reference can be ab-
solute or relative, as in these examples:
(...(absolute)...)...(?2)...
@@ -9924,35 +10076,75 @@ GROUPS AS SUBROUTINES
(sens|respons)e and \1ibility
- matches "sense and sensibility" and "response and responsibility", but
+ matches "sense and sensibility" and "response and responsibility", but
not "sense and responsibility". If instead the pattern
(sens|respons)e and (?1)ibility
- is used, it does match "sense and responsibility" as well as the other
- two strings. Another example is given in the discussion of DEFINE
+ is used, it does match "sense and responsibility" as well as the other
+ two strings. Another example is given in the discussion of DEFINE
above.
- Like recursions, subroutine calls used to be treated as atomic, but
- this changed at PCRE2 release 10.30, so backtracking into subroutine
- calls can now occur. However, any capturing parentheses that are set
+ Like recursions, subroutine calls used to be treated as atomic, but
+ this changed at PCRE2 release 10.30, so backtracking into subroutine
+ calls can now occur. However, any capturing parentheses that are set
during the subroutine call revert to their previous values afterwards.
- Processing options such as case-independence are fixed when a group is
- defined, so if it is used as a subroutine, such options cannot be
+ Processing options such as case-independence are fixed when a group is
+ defined, so if it is used as a subroutine, such options cannot be
changed for different calls. For example, consider this pattern:
(abc)(?i:(?-1))
- It matches "abcabc". It does not match "abcABC" because the change of
+ It matches "abcabc". It does not match "abcABC" because the change of
processing option does not affect the called group.
- The behaviour of backtracking control verbs in groups when called as
+ The behaviour of backtracking control verbs in groups when called as
subroutines is described in the section entitled "Backtracking verbs in
subroutines" below.
+ Recursion and subroutines with returned capture groups
-ONIGURUMA SUBROUTINE SYNTAX
+ Since PCRE2 10.46, recursion and subroutine calls may also specify a
+ list of capture groups to return. This is a PCRE2 syntax extension not
+ supported by Perl. The pattern matching recurses into the referenced
+ expression as described above, however, when the recursion returns to
+ the calling expression the subgroups captured during the recursion can
+ be retained when the calling expression's context is restored.
+
+ When used as a subroutine, this allows the subroutine's capture groups
+ to be used as return values.
+
+ Only the specific capture groups listed by the caller will be retained,
+ using the following syntax:
+
+ (?R(grouplist)) recurse whole pattern, returning capture groups
+ (?n(grouplist)) )
+ (?+n(grouplist)) )
+ (?-n(grouplist)) ) call subroutine, returning capture groups
+ (?&name(grouplist)) )
+ (?P>name(grouplist)) )
+
+ The list of capture groups "grouplist" is a comma-separated list of
+ (absolute or relative) group numbers, and group names enclosed in sin-
+ gle quotes or angle brackets.
+
+ Here is an example which first uses the DEFINE condition to create a
+ re-usable routine for matching a weekday, then calls that subroutine
+ and retains the groups it captures for use later:
+
+ (?x: # ignore whitespace for clarity
+ # Define the routine "weekendday" which matches Saturday or
+ # Sunday, and returns the Sat/Sun prefix as \k.
+ (?(DEFINE) (?
+ (?|(?Sat)urday|(?Sun)day) ) )
+ # Call the routine. Matches "Saturday,Sat" or "Sunday,Sun".
+ (?&weekendday()),\k )
+
+ This feature is not available using the Oniguruma syntax \g<...> or
+ \g'...' below.
+
+ Oniguruma subroutine syntax
For compatibility with Oniguruma, the non-Perl syntax \g followed by a
name or a number enclosed either in angle brackets or single quotes, is
@@ -10070,10 +10262,10 @@ BACKTRACKING CONTROL
A closing parenthesis can be included in a name either as \) or between
\Q and \E. In addition to backslash processing, if the PCRE2_EXTENDED
- or PCRE2_EXTENDED_MORE option is also set, unescaped whitespace in verb
- names is skipped, and #-comments are recognized, exactly as in the rest
- of the pattern. PCRE2_EXTENDED and PCRE2_EXTENDED_MORE do not affect
- verb names unless PCRE2_ALT_VERBNAMES is also set.
+ or PCRE2_EXTENDED_MORE option is also set, unescaped white space in
+ verb names is skipped, and #-comments are recognized, exactly as in the
+ rest of the pattern. PCRE2_EXTENDED and PCRE2_EXTENDED_MORE do not af-
+ fect verb names unless PCRE2_ALT_VERBNAMES is also set.
The maximum length of a name is 255 in the 8-bit library and 65535 in
the 16-bit and 32-bit libraries. If the name is empty, that is, if the
@@ -10590,11 +10782,11 @@ AUTHOR
REVISION
- Last updated: 27 November 2024
+ Last updated: 03 September 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 27 November 2024 PCRE2PATTERN(3)
+PCRE2 10.47 03 September 2025 PCRE2PATTERN(3)
------------------------------------------------------------------------------
@@ -10849,7 +11041,7 @@ REVISION
Copyright (c) 1997-2022 University of Cambridge.
-PCRE2 10.46 06 December 2022 PCRE2PERFORM(3)
+PCRE2 10.47 06 December 2022 PCRE2PERFORM(3)
------------------------------------------------------------------------------
@@ -11207,7 +11399,7 @@ REVISION
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 27 November 2024 PCRE2POSIX(3)
+PCRE2 10.47 27 November 2024 PCRE2POSIX(3)
------------------------------------------------------------------------------
@@ -11224,7 +11416,7 @@ PCRE2 SAMPLE PROGRAM
PCRE2 is supplied in the file pcre2demo.c in the src directory in the
PCRE2 distribution. A listing of this program is given in the pcre2demo
documentation. If you do not have a copy of the PCRE2 distribution, you
- can save this listing to re-create the contents of pcre2demo.c.
+ can save this listing to recreate the contents of pcre2demo.c.
The demonstration program compiles the regular expression that is its
first argument, and matches it against the subject string in its second
@@ -11265,6 +11457,7 @@ PCRE2 SAMPLE PROGRAM
./pcre2demo 'cat|dog' 'the cat sat on the mat'
./pcre2demo -g 'cat|dog' 'the dog sat on the cat'
+ ./pcre2demo -i 'cat' 'the dog sat on the CAT'
Note that there is a much more comprehensive test program, called
pcre2test, which supports many more facilities for testing regular ex-
@@ -11296,11 +11489,11 @@ AUTHOR
REVISION
- Last updated: 14 November 2023
+ Last updated: 28 February 2025
Copyright (c) 1997-2016 University of Cambridge.
-PCRE2 10.46 14 November 2023 PCRE2SAMPLE(3)
+PCRE2 10.47 28 February 2025 PCRE2SAMPLE(3)
------------------------------------------------------------------------------
PCRE2SERIALIZE(3) Library Functions Manual PCRE2SERIALIZE(3)
@@ -11491,7 +11684,7 @@ REVISION
Copyright (c) 1997-2018 University of Cambridge.
-PCRE2 10.46 19 January 2024 PCRE2SERIALIZE(3)
+PCRE2 10.47 19 January 2024 PCRE2SERIALIZE(3)
------------------------------------------------------------------------------
@@ -11777,7 +11970,7 @@ CHARACTER CLASSES
PERL EXTENDED CHARACTER CLASSES
(?[...]) Perl extended character class
- (?[\p{Thai} & \p{Nd}]) operators; whitespace ignored
+ (?[\p{Thai} & \p{Nd}]) operators; white space ignored
(?[(x - y) & z]) parentheses for grouping
(?[ [^3] & \p{Nd} ]) [...] is a nested ordinary class
@@ -12013,8 +12206,8 @@ SUBSTRING SCAN ASSERTION
(*scan_substring:(grouplist)...) scan captured substring
(*scs:(grouplist)...) scan captured substring
- The comma-separated list may identify groups in any of the following
- ways:
+ The comma-separated list "grouplist" may identify groups in any of the
+ following ways:
n absolute reference
+n relative reference
@@ -12065,29 +12258,56 @@ SUBROUTINE REFERENCES (POSSIBLY RECURSIVE)
\g<-n> call subroutine by relative number (PCRE2 extension)
\g'-n' call subroutine by relative number (PCRE2 extension)
+ The variants using parentheses (?...) may also specify a list of cap-
+ ture groups to return, which shall be retained in the calling subex-
+ pression if set during the recursion (this feature is not supported by
+ Perl).
+
+ (?R(grouplist)) recurse whole pattern, returning capture groups
+ (PCRE2 extension)
+ (?n(grouplist)) )
+ (?+n(grouplist)) ) call subroutine, returning capture groups
+ (?-n(grouplist)) ) (PCRE2 extension)
+ (?&name(grouplist)) )
+ (?P>name(grouplist)) )
+
+ The comma-separated list "grouplist" uses the same syntax as
+ (*scan_substring:(grouplist)...), and may identify groups in any of the
+ following ways:
+
+ n absolute reference
+ +n relative reference
+ -n relative reference
+ name
+ 'name' name
+
CONDITIONAL PATTERNS
(?(condition)yes-pattern)
(?(condition)yes-pattern|no-pattern)
- (?(n) absolute reference condition
- (?(+n) relative reference condition (PCRE2 extension)
- (?(-n) relative reference condition (PCRE2 extension)
- (?() named reference condition (Perl)
- (?('name') named reference condition (Perl)
- (?(name) named reference condition (PCRE2, deprecated)
- (?(R) overall recursion condition
- (?(Rn) specific numbered group recursion condition
- (?(R&name) specific named group recursion condition
- (?(DEFINE) define groups for reference
- (?(VERSION[>]=n.m) test PCRE2 version
- (?(assert) assertion condition
+ (?(n) absolute reference condition
+ (?(+n) relative reference condition (PCRE2 extension)
+ (?(-n) relative reference condition (PCRE2 extension)
+ (?() named reference condition (Perl)
+ (?('name') named reference condition (Perl)
+ (?(name) named reference condition (PCRE2, deprecated)
+ (?(R) overall recursion condition
+ (?(Rn) specific numbered group recursion condition
+ (?(R&name) specific named group recursion condition
+ (?(DEFINE) define groups for reference
+ (?(VERSION[>]=n[.m]) test PCRE2 version
+ (?(assert) assertion condition
Note the ambiguity of (?(R) and (?(Rn) which might be named reference
conditions or recursion tests. Such a condition is interpreted as a
reference condition if the relevant named group exists.
+ The parts within brackets for the VERSION conditional syntax could be
+ ommited. The fractional part of the version number defaults to 0 in
+ that case.
+
BACKTRACKING CONTROL
@@ -12143,25 +12363,26 @@ REPLACEMENT STRINGS
$` insert the substring that precedes the match
$' insert the substring that follows the match
$_ insert the entire input string
+ $+ insert the highest-numbered capture group which
+ matched
$*MARK or ${*MARK} insert a control verb name
- For ${n}, n can be a name or a number. If PCRE2_SUBSTITUTE_EXTENDED is
+ For ${n}, n can be a name or a number. If PCRE2_SUBSTITUTE_EXTENDED is
set, there is additional interpretation:
- 1. Backslash is an escape character, and the forms described in "ES-
+ 1. Backslash is an escape character, and the forms described in "ES-
CAPED CHARACTERS" above are recognized. Also:
- \Q...\E can be used to suppress interpretation
- \l force the next character to lower case
- \u force the next character to upper case
- \L force subsequent characters to lower case
- \U force subsequent characters to upper case
- \u\L force next character to upper case, then all lower
- \l\U force next character to lower case, then all upper
- \E end \L or \U case forcing
- \b backspace character (note: as in character class in pat-
- tern)
- \v vertical tab character (note: not the same as in a pattern)
+ \Q...\E can be used to suppress interpretation
+ \l force the next character to lower case
+ \u force the next character to upper case
+ \L force subsequent characters to lower case
+ \U force subsequent characters to upper case
+ \u\L force next character to upper case, then all lower
+ \l\U force next character to lower case, then all upper
+ \E end \L or \U case forcing
+ \b backspace character (note: as in character class in pattern)
+ \v vertical tab character (note: not the same as in a pattern)
2. The Python form \g, where the angle brackets are part of the syn-
tax and n is either a group name or a number, is recognized as an al-
@@ -12191,11 +12412,11 @@ AUTHOR
REVISION
- Last updated: 27 November 2024
+ Last updated: 14 October 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 27 November 2024 PCRE2SYNTAX(3)
+PCRE2 10.47 14 October 2025 PCRE2SYNTAX(3)
------------------------------------------------------------------------------
@@ -12690,7 +12911,7 @@ REVISION
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 27 November 2024 PCRE2UNICODE(3)
+PCRE2 10.47 27 November 2024 PCRE2UNICODE(3)
------------------------------------------------------------------------------
diff --git a/doc/pcre2_callout_enumerate.3 b/doc/pcre2_callout_enumerate.3
index 0c24114..9065fd8 100644
--- a/doc/pcre2_callout_enumerate.3
+++ b/doc/pcre2_callout_enumerate.3
@@ -1,4 +1,4 @@
-.TH PCRE2_COMPILE 3 "23 March 2017" "PCRE2 10.46"
+.TH PCRE2_COMPILE 3 "23 March 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_code_copy.3 b/doc/pcre2_code_copy.3
index f7a2d78..9b6526f 100644
--- a/doc/pcre2_code_copy.3
+++ b/doc/pcre2_code_copy.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CODE_COPY 3 "22 November 2016" "PCRE2 10.46"
+.TH PCRE2_CODE_COPY 3 "22 November 2016" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_code_copy_with_tables.3 b/doc/pcre2_code_copy_with_tables.3
index ebde43b..3896577 100644
--- a/doc/pcre2_code_copy_with_tables.3
+++ b/doc/pcre2_code_copy_with_tables.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CODE_COPY 3 "16 January 2017" "PCRE2 10.46"
+.TH PCRE2_CODE_COPY 3 "16 January 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_code_free.3 b/doc/pcre2_code_free.3
index 101d395..ca660e8 100644
--- a/doc/pcre2_code_free.3
+++ b/doc/pcre2_code_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CODE_FREE 3 "28 June 2018" "PCRE2 10.46"
+.TH PCRE2_CODE_FREE 3 "28 June 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_compile.3 b/doc/pcre2_compile.3
index e2dab62..eb40814 100644
--- a/doc/pcre2_compile.3
+++ b/doc/pcre2_compile.3
@@ -1,4 +1,4 @@
-.TH PCRE2_COMPILE 3 "30 October 2024" "PCRE2 10.46"
+.TH PCRE2_COMPILE 3 "30 October 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_compile_context_copy.3 b/doc/pcre2_compile_context_copy.3
index 94a0c6e..c3353b5 100644
--- a/doc/pcre2_compile_context_copy.3
+++ b/doc/pcre2_compile_context_copy.3
@@ -1,4 +1,4 @@
-.TH PCRE2_COMPILE_CONTEXT_COPY 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_COMPILE_CONTEXT_COPY 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_compile_context_create.3 b/doc/pcre2_compile_context_create.3
index 9dcdaa3..f094977 100644
--- a/doc/pcre2_compile_context_create.3
+++ b/doc/pcre2_compile_context_create.3
@@ -1,4 +1,4 @@
-.TH PCRE2_COMPILE_CONTEXT_CREATE 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_COMPILE_CONTEXT_CREATE 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_compile_context_free.3 b/doc/pcre2_compile_context_free.3
index c9dd260..ded964e 100644
--- a/doc/pcre2_compile_context_free.3
+++ b/doc/pcre2_compile_context_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_COMPILE_CONTEXT_FREE 3 "28 June 2018" "PCRE2 10.46"
+.TH PCRE2_COMPILE_CONTEXT_FREE 3 "28 June 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_config.3 b/doc/pcre2_config.3
index 37c1597..e109180 100644
--- a/doc/pcre2_config.3
+++ b/doc/pcre2_config.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CONFIG 3 "16 September 2017" "PCRE2 10.46"
+.TH PCRE2_CONFIG 3 "03 September 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -6,8 +6,9 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.sp
.B #include
.PP
-.SM
+.nf
.B int pcre2_config(uint32_t \fIwhat\fP, void *\fIwhere\fP);
+.fi
.
.SH DESCRIPTION
.rs
@@ -28,36 +29,37 @@ PCRE2_CONFIG_UNICODE_VERSION, and PCRE2_CONFIG_VERSION it must point to a
buffer that is large enough to hold the string. For all other codes it must
point to a uint32_t integer variable. The available codes are:
.sp
- PCRE2_CONFIG_BSR Indicates what \eR matches by default:
- PCRE2_BSR_UNICODE
- PCRE2_BSR_ANYCRLF
- PCRE2_CONFIG_COMPILED_WIDTHS Which of 8/16/32 support was compiled
- PCRE2_CONFIG_DEPTHLIMIT Default backtracking depth limit
- PCRE2_CONFIG_HEAPLIMIT Default heap memory limit
+ PCRE2_CONFIG_BSR Indicates what \eR matches by default:
+ PCRE2_BSR_UNICODE
+ PCRE2_BSR_ANYCRLF
+ PCRE2_CONFIG_COMPILED_WIDTHS Which of 8/16/32 support was compiled
+ PCRE2_CONFIG_DEPTHLIMIT Default backtracking depth limit
+ PCRE2_CONFIG_EFFECTIVE_LINKSIZE How many bytes are used for link size
+ PCRE2_CONFIG_HEAPLIMIT Default heap memory limit
.\" JOIN
- PCRE2_CONFIG_JIT Availability of just-in-time compiler
- support (1=yes 0=no)
+ PCRE2_CONFIG_JIT Availability of just-in-time compiler
+ support (1=yes 0=no)
.\" JOIN
- PCRE2_CONFIG_JITTARGET Information (a string) about the target
- architecture for the JIT compiler
- PCRE2_CONFIG_LINKSIZE Configured internal link size (2, 3, 4)
- PCRE2_CONFIG_MATCHLIMIT Default internal resource limit
+ PCRE2_CONFIG_JITTARGET Information (a string) about the target
+ architecture for the JIT compiler
+ PCRE2_CONFIG_LINKSIZE Configured internal link size (2, 3, 4)
+ PCRE2_CONFIG_MATCHLIMIT Default internal resource limit
PCRE2_CONFIG_NEVER_BACKSLASH_C Whether or not \eC is disabled
- PCRE2_CONFIG_NEWLINE Code for the default newline sequence:
- PCRE2_NEWLINE_CR
- PCRE2_NEWLINE_LF
- PCRE2_NEWLINE_CRLF
- PCRE2_NEWLINE_ANY
- PCRE2_NEWLINE_ANYCRLF
- PCRE2_NEWLINE_NUL
- PCRE2_CONFIG_PARENSLIMIT Default parentheses nesting limit
- PCRE2_CONFIG_RECURSIONLIMIT Obsolete: use PCRE2_CONFIG_DEPTHLIMIT
- PCRE2_CONFIG_STACKRECURSE Obsolete: always returns 0
+ PCRE2_CONFIG_NEWLINE Code for the default newline sequence:
+ PCRE2_NEWLINE_CR
+ PCRE2_NEWLINE_LF
+ PCRE2_NEWLINE_CRLF
+ PCRE2_NEWLINE_ANY
+ PCRE2_NEWLINE_ANYCRLF
+ PCRE2_NEWLINE_NUL
+ PCRE2_CONFIG_PARENSLIMIT Default parentheses nesting limit
+ PCRE2_CONFIG_RECURSIONLIMIT Obsolete: use PCRE2_CONFIG_DEPTHLIMIT
+ PCRE2_CONFIG_STACKRECURSE Obsolete: always returns 0
.\" JOIN
- PCRE2_CONFIG_UNICODE Availability of Unicode support (1=yes
- 0=no)
- PCRE2_CONFIG_UNICODE_VERSION The Unicode version (a string)
- PCRE2_CONFIG_VERSION The PCRE2 version (a string)
+ PCRE2_CONFIG_UNICODE Availability of Unicode support
+ (1=yes 0=no)
+ PCRE2_CONFIG_UNICODE_VERSION The Unicode version (a string)
+ PCRE2_CONFIG_VERSION The PCRE2 version (a string)
.sp
The function yields a non-negative value on success or the negative value
PCRE2_ERROR_BADOPTION otherwise. This is also the result for the
diff --git a/doc/pcre2_convert_context_copy.3 b/doc/pcre2_convert_context_copy.3
index 5f91691..32073f0 100644
--- a/doc/pcre2_convert_context_copy.3
+++ b/doc/pcre2_convert_context_copy.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CONVERT_CONTEXT_COPY 3 "12 July 2017" "PCRE2 10.46"
+.TH PCRE2_CONVERT_CONTEXT_COPY 3 "12 July 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_convert_context_create.3 b/doc/pcre2_convert_context_create.3
index 73ccff4..6a5c9be 100644
--- a/doc/pcre2_convert_context_create.3
+++ b/doc/pcre2_convert_context_create.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CONVERT_CONTEXT_CREATE 3 "12 July 2017" "PCRE2 10.46"
+.TH PCRE2_CONVERT_CONTEXT_CREATE 3 "12 July 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_convert_context_free.3 b/doc/pcre2_convert_context_free.3
index 6ae3541..7fbda34 100644
--- a/doc/pcre2_convert_context_free.3
+++ b/doc/pcre2_convert_context_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CONVERT_CONTEXT_FREE 3 "13 August 2018" "PCRE2 10.46"
+.TH PCRE2_CONVERT_CONTEXT_FREE 3 "13 August 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_converted_pattern_free.3 b/doc/pcre2_converted_pattern_free.3
index d0c60ca..3b09fe7 100644
--- a/doc/pcre2_converted_pattern_free.3
+++ b/doc/pcre2_converted_pattern_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_CONVERTED_PATTERN_FREE 3 "13 August 2018" "PCRE2 10.46"
+.TH PCRE2_CONVERTED_PATTERN_FREE 3 "13 August 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_dfa_match.3 b/doc/pcre2_dfa_match.3
index 299a7f8..4a4673b 100644
--- a/doc/pcre2_dfa_match.3
+++ b/doc/pcre2_dfa_match.3
@@ -1,4 +1,4 @@
-.TH PCRE2_DFA_MATCH 3 "31 August 2021" "PCRE2 10.46"
+.TH PCRE2_DFA_MATCH 3 "31 August 2021" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_general_context_copy.3 b/doc/pcre2_general_context_copy.3
index a852018..1f73772 100644
--- a/doc/pcre2_general_context_copy.3
+++ b/doc/pcre2_general_context_copy.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GENERAL_CONTEXT_COPY 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_GENERAL_CONTEXT_COPY 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_general_context_create.3 b/doc/pcre2_general_context_create.3
index 445ca38..cd9e1b5 100644
--- a/doc/pcre2_general_context_create.3
+++ b/doc/pcre2_general_context_create.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GENERAL_CONTEXT_CREATE 3 "23 January 2023" "PCRE2 10.46"
+.TH PCRE2_GENERAL_CONTEXT_CREATE 3 "23 January 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_general_context_free.3 b/doc/pcre2_general_context_free.3
index 83cd557..3d69d13 100644
--- a/doc/pcre2_general_context_free.3
+++ b/doc/pcre2_general_context_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GENERAL_CONTEXT_FREE 3 "28 June 2018" "PCRE2 10.46"
+.TH PCRE2_GENERAL_CONTEXT_FREE 3 "28 June 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_get_error_message.3 b/doc/pcre2_get_error_message.3
index d86a671..cffc573 100644
--- a/doc/pcre2_get_error_message.3
+++ b/doc/pcre2_get_error_message.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GET_ERROR_MESSAGE 3 "24 March 2017" "PCRE2 10.46"
+.TH PCRE2_GET_ERROR_MESSAGE 3 "24 March 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_get_mark.3 b/doc/pcre2_get_mark.3
index fd0725c..6d29106 100644
--- a/doc/pcre2_get_mark.3
+++ b/doc/pcre2_get_mark.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GET_MARK 3 "13 January 2018" "PCRE2 10.46"
+.TH PCRE2_GET_MARK 3 "13 January 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_get_match_data_heapframes_size.3 b/doc/pcre2_get_match_data_heapframes_size.3
index 64a07f9..555bcd2 100644
--- a/doc/pcre2_get_match_data_heapframes_size.3
+++ b/doc/pcre2_get_match_data_heapframes_size.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GET_MATCH_DATA_HEAPFRAMES_SIZE 3 "18 January 2023" "PCRE2 10.46"
+.TH PCRE2_GET_MATCH_DATA_HEAPFRAMES_SIZE 3 "18 January 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_get_match_data_size.3 b/doc/pcre2_get_match_data_size.3
index e5ba402..3ebc4fd 100644
--- a/doc/pcre2_get_match_data_size.3
+++ b/doc/pcre2_get_match_data_size.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GET_MATCH_DATA_SIZE 3 "17 October 2019" "PCRE2 10.46"
+.TH PCRE2_GET_MATCH_DATA_SIZE 3 "17 October 2019" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_get_ovector_count.3 b/doc/pcre2_get_ovector_count.3
index c41ed7f..23f072d 100644
--- a/doc/pcre2_get_ovector_count.3
+++ b/doc/pcre2_get_ovector_count.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GET_OVECTOR_COUNT 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_GET_OVECTOR_COUNT 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_get_ovector_pointer.3 b/doc/pcre2_get_ovector_pointer.3
index 9cf5e99..7d9ec1e 100644
--- a/doc/pcre2_get_ovector_pointer.3
+++ b/doc/pcre2_get_ovector_pointer.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GET_OVECTOR_POINTER 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_GET_OVECTOR_POINTER 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_get_startchar.3 b/doc/pcre2_get_startchar.3
index 7c99df4..b9c4f7b 100644
--- a/doc/pcre2_get_startchar.3
+++ b/doc/pcre2_get_startchar.3
@@ -1,4 +1,4 @@
-.TH PCRE2_GET_STARTCHAR 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_GET_STARTCHAR 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_jit_compile.3 b/doc/pcre2_jit_compile.3
index 53cea8b..9961d47 100644
--- a/doc/pcre2_jit_compile.3
+++ b/doc/pcre2_jit_compile.3
@@ -1,4 +1,4 @@
-.TH PCRE2_JIT_COMPILE 3 "22 August 2024" "PCRE2 10.46"
+.TH PCRE2_JIT_COMPILE 3 "22 August 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_jit_free_unused_memory.3 b/doc/pcre2_jit_free_unused_memory.3
index 3eb23db..fa1073a 100644
--- a/doc/pcre2_jit_free_unused_memory.3
+++ b/doc/pcre2_jit_free_unused_memory.3
@@ -1,4 +1,4 @@
-.TH PCRE2_JIT_FREE_UNUSED_MEMORY 3 "24 April 2020" "PCRE2 10.46"
+.TH PCRE2_JIT_FREE_UNUSED_MEMORY 3 "24 April 2020" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_jit_match.3 b/doc/pcre2_jit_match.3
index 60e3a63..e147b90 100644
--- a/doc/pcre2_jit_match.3
+++ b/doc/pcre2_jit_match.3
@@ -1,4 +1,4 @@
-.TH PCRE2_JIT_MATCH 3 "20 January 2023" "PCRE2 10.46"
+.TH PCRE2_JIT_MATCH 3 "20 January 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_jit_stack_assign.3 b/doc/pcre2_jit_stack_assign.3
index f3a0a04..61fc963 100644
--- a/doc/pcre2_jit_stack_assign.3
+++ b/doc/pcre2_jit_stack_assign.3
@@ -1,4 +1,4 @@
-.TH PCRE2_JIT_STACK_ASSIGN 3 "13 August 2018" "PCRE2 10.46"
+.TH PCRE2_JIT_STACK_ASSIGN 3 "13 August 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_jit_stack_create.3 b/doc/pcre2_jit_stack_create.3
index 3ad2adc..494eee8 100644
--- a/doc/pcre2_jit_stack_create.3
+++ b/doc/pcre2_jit_stack_create.3
@@ -1,4 +1,4 @@
-.TH PCRE2_JIT_STACK_CREATE 3 "23 January 2023" "PCRE2 10.46"
+.TH PCRE2_JIT_STACK_CREATE 3 "23 January 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_jit_stack_free.3 b/doc/pcre2_jit_stack_free.3
index 8f1d5a4..0b88759 100644
--- a/doc/pcre2_jit_stack_free.3
+++ b/doc/pcre2_jit_stack_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_JIT_STACK_FREE 3 "13 August 2018" "PCRE2 10.46"
+.TH PCRE2_JIT_STACK_FREE 3 "26 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -6,8 +6,9 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.sp
.B #include
.PP
-.SM
+.nf
.B void pcre2_jit_stack_free(pcre2_jit_stack *\fIjit_stack\fP);
+.fi
.
.SH DESCRIPTION
.rs
diff --git a/doc/pcre2_maketables.3 b/doc/pcre2_maketables.3
index d859b31..b42933f 100644
--- a/doc/pcre2_maketables.3
+++ b/doc/pcre2_maketables.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MAKETABLES 3 "28 July 2019" "PCRE2 10.46"
+.TH PCRE2_MAKETABLES 3 "26 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -6,8 +6,9 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.sp
.B #include
.PP
-.SM
+.nf
.B const uint8_t *pcre2_maketables(pcre2_general_context *\fIgcontext\fP);
+.fi
.
.SH DESCRIPTION
.rs
diff --git a/doc/pcre2_maketables_free.3 b/doc/pcre2_maketables_free.3
index 44a8db7..4b835c3 100644
--- a/doc/pcre2_maketables_free.3
+++ b/doc/pcre2_maketables_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MAKETABLES_FREE 3 "03 September 2019" "PCRE2 10.46"
+.TH PCRE2_MAKETABLES_FREE 3 "03 September 2019" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_match.3 b/doc/pcre2_match.3
index 50cd1f2..5701b75 100644
--- a/doc/pcre2_match.3
+++ b/doc/pcre2_match.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MATCH 3 "27 January 2024" "PCRE2 10.46"
+.TH PCRE2_MATCH 3 "27 January 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_match_context_copy.3 b/doc/pcre2_match_context_copy.3
index 88614b1..cbce67a 100644
--- a/doc/pcre2_match_context_copy.3
+++ b/doc/pcre2_match_context_copy.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MATCH_CONTEXT_COPY 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_MATCH_CONTEXT_COPY 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_match_context_create.3 b/doc/pcre2_match_context_create.3
index b20c702..ab624a3 100644
--- a/doc/pcre2_match_context_create.3
+++ b/doc/pcre2_match_context_create.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MATCH_CONTEXT_CREATE 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_MATCH_CONTEXT_CREATE 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_match_context_free.3 b/doc/pcre2_match_context_free.3
index 83c66a3..e0a4d62 100644
--- a/doc/pcre2_match_context_free.3
+++ b/doc/pcre2_match_context_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MATCH_CONTEXT_FREE 3 "28 June 2018" "PCRE2 10.46"
+.TH PCRE2_MATCH_CONTEXT_FREE 3 "28 June 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_match_data_create.3 b/doc/pcre2_match_data_create.3
index 59c9de8..49ee289 100644
--- a/doc/pcre2_match_data_create.3
+++ b/doc/pcre2_match_data_create.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MATCH_DATA_CREATE 3 "28 August 2021" "PCRE2 10.46"
+.TH PCRE2_MATCH_DATA_CREATE 3 "28 August 2021" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_match_data_create_from_pattern.3 b/doc/pcre2_match_data_create_from_pattern.3
index ec9a62f..d105b62 100644
--- a/doc/pcre2_match_data_create_from_pattern.3
+++ b/doc/pcre2_match_data_create_from_pattern.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MATCH_DATA_CREATE_FROM_PATTERN 3 "19 August 2022" "PCRE2 10.46"
+.TH PCRE2_MATCH_DATA_CREATE_FROM_PATTERN 3 "11 August 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -15,19 +15,19 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.rs
.sp
This function creates a new match data block for holding the result of a match.
-The first argument points to a compiled pattern. The number of capturing
-parentheses within the pattern is used to compute the number of pairs of
-offsets that are required in the match data block. These form the "output
-vector" (ovector) within the match data block, and are used to identify the
-matched string and any captured substrings when matching with
-\fBpcre2_match()\fP. If you are using \fBpcre2_dfa_match()\fP, which uses the
-output vector in a different way, you should use \fBpcre2_match_data_create()\fP
-instead of this function.
+If the first argument is NULL, this function returns NULL, otherwise the first
+argument points to a compiled pattern. The number of capturing parentheses
+within the pattern is used to compute the number of pairs of offsets that are
+required in the match data block. These form the "output vector" (ovector)
+within the match data block, and are used to identify the matched string and
+any captured substrings when matching with \fBpcre2_match()\fP. If you are
+using \fBpcre2_dfa_match()\fP, which uses the output vector in a different way,
+you should use \fBpcre2_match_data_create()\fP instead of this function.
.P
The second argument points to a general context, for custom memory management,
-or is NULL to use the same memory allocator as was used for the compiled
+or is NULL to use the same memory allocator that was used for the compiled
pattern. The result of the function is NULL if the memory for the block could
-not be obtained.
+not be obtained or if NULL was provided as the first argument.
.P
There is a complete description of the PCRE2 native API in the
.\" HREF
diff --git a/doc/pcre2_match_data_free.3 b/doc/pcre2_match_data_free.3
index 855ddce..245b3e2 100644
--- a/doc/pcre2_match_data_free.3
+++ b/doc/pcre2_match_data_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_MATCH_DATA_FREE 3 "16 August 2023" "PCRE2 10.46"
+.TH PCRE2_MATCH_DATA_FREE 3 "16 August 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_next_match.3 b/doc/pcre2_next_match.3
new file mode 100644
index 0000000..4c8f4e9
--- /dev/null
+++ b/doc/pcre2_next_match.3
@@ -0,0 +1,41 @@
+.TH PCRE2_NEXT_MATCH 3 "24 March 2025" "PCRE2 10.47"
+.SH NAME
+PCRE2 - Perl-compatible regular expressions (revised API)
+.SH SYNOPSIS
+.rs
+.sp
+.B #include
+.PP
+.nf
+.B int pcre2_next_match(pcre2_match_data *\fImatch_data\fP,
+.B " PCRE2_SIZE *\fIpstart_offset\fP, uint32_t *\fIpoptions\fP);"
+.fi
+.
+.SH DESCRIPTION
+.rs
+.sp
+This function can be called after one of the match functions
+(\fBpcre2_match()\fP, \fBpcre2_dfa_match()\fP, or \fBpcre2_jit_match()\fP), and
+must be provided with the same \fImatch_data\fP parameter. It outputs the
+appropriate parameters for searching for the next match in the same subject
+string, and is suitable for applications providing "global" matching behaviour
+(for example, replacing all matches in the subject, or splitting the subject on
+all matches, or simply counting the number of matches).
+.P
+It returns 0 ("false") if there is no need to make any further match attempts,
+or 1 ("true") if another match should be attempted.
+.P
+The *\fIpstart_offset\fP and *\fIpoptions\fP are set if the function returns 1.
+The *\fIpstart_offset\fP should be passed to the next match attempt directly,
+and the *\fIpoptions\fP should be passed to the next match attempt by combining
+with the application's match options using OR.
+.P
+There is a complete description of the PCRE2 native API in the
+.\" HREF
+\fBpcre2api\fP
+.\"
+page and a description of the POSIX API in the
+.\" HREF
+\fBpcre2posix\fP
+.\"
+page.
diff --git a/doc/pcre2_pattern_convert.3 b/doc/pcre2_pattern_convert.3
index f3266f2..4302c52 100644
--- a/doc/pcre2_pattern_convert.3
+++ b/doc/pcre2_pattern_convert.3
@@ -1,4 +1,4 @@
-.TH PCRE2_PATTERN_CONVERT 3 "12 July 2017" "PCRE2 10.46"
+.TH PCRE2_PATTERN_CONVERT 3 "12 July 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_pattern_info.3 b/doc/pcre2_pattern_info.3
index 05e133e..aa8c662 100644
--- a/doc/pcre2_pattern_info.3
+++ b/doc/pcre2_pattern_info.3
@@ -1,4 +1,4 @@
-.TH PCRE2_PATTERN_INFO 3 "14 February 2019" "PCRE2 10.46"
+.TH PCRE2_PATTERN_INFO 3 "14 February 2019" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_serialize_decode.3 b/doc/pcre2_serialize_decode.3
index b81cf7e..363d73c 100644
--- a/doc/pcre2_serialize_decode.3
+++ b/doc/pcre2_serialize_decode.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SERIALIZE_DECODE 3 "22 April 2022" "PCRE2 10.46"
+.TH PCRE2_SERIALIZE_DECODE 3 "22 April 2022" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_serialize_encode.3 b/doc/pcre2_serialize_encode.3
index 9fb7aed..ca0ff3b 100644
--- a/doc/pcre2_serialize_encode.3
+++ b/doc/pcre2_serialize_encode.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SERIALIZE_ENCODE 3 "13 August 2018" "PCRE2 10.46"
+.TH PCRE2_SERIALIZE_ENCODE 3 "13 August 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_serialize_free.3 b/doc/pcre2_serialize_free.3
index c90378d..fc563de 100644
--- a/doc/pcre2_serialize_free.3
+++ b/doc/pcre2_serialize_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SERIALIZE_FREE 3 "13 August 2018" "PCRE2 10.46"
+.TH PCRE2_SERIALIZE_FREE 3 "13 August 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_serialize_get_number_of_codes.3 b/doc/pcre2_serialize_get_number_of_codes.3
index 689771f..9cb0fbb 100644
--- a/doc/pcre2_serialize_get_number_of_codes.3
+++ b/doc/pcre2_serialize_get_number_of_codes.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SERIALIZE_GET_NUMBER_OF_CODES 3 "13 August 2018" "PCRE2 10.46"
+.TH PCRE2_SERIALIZE_GET_NUMBER_OF_CODES 3 "13 August 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_bsr.3 b/doc/pcre2_set_bsr.3
index 6fdf798..a0d6014 100644
--- a/doc/pcre2_set_bsr.3
+++ b/doc/pcre2_set_bsr.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_BSR 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_SET_BSR 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_callout.3 b/doc/pcre2_set_callout.3
index 6e8334c..a71dd7e 100644
--- a/doc/pcre2_set_callout.3
+++ b/doc/pcre2_set_callout.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_CALLOUT 3 "25 March 2017" "PCRE2 10.46"
+.TH PCRE2_SET_CALLOUT 3 "25 March 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_character_tables.3 b/doc/pcre2_set_character_tables.3
index 252aa48..e6d0503 100644
--- a/doc/pcre2_set_character_tables.3
+++ b/doc/pcre2_set_character_tables.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_CHARACTER_TABLES 3 "15 April 2020" "PCRE2 10.46"
+.TH PCRE2_SET_CHARACTER_TABLES 3 "15 April 2020" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_compile_extra_options.3 b/doc/pcre2_set_compile_extra_options.3
index 37e2d4c..56e8e00 100644
--- a/doc/pcre2_set_compile_extra_options.3
+++ b/doc/pcre2_set_compile_extra_options.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_COMPILE_EXTRA_OPTIONS 3 "14 October 2024" "PCRE2 10.46"
+.TH PCRE2_SET_COMPILE_EXTRA_OPTIONS 3 "14 October 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_compile_recursion_guard.3 b/doc/pcre2_set_compile_recursion_guard.3
index e6126cb..44ed685 100644
--- a/doc/pcre2_set_compile_recursion_guard.3
+++ b/doc/pcre2_set_compile_recursion_guard.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_COMPILE_RECURSION_GUARD 3 "26 November 2014" "PCRE2 10.46"
+.TH PCRE2_SET_COMPILE_RECURSION_GUARD 3 "26 November 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_depth_limit.3 b/doc/pcre2_set_depth_limit.3
index 5ae81db..a816a0c 100644
--- a/doc/pcre2_set_depth_limit.3
+++ b/doc/pcre2_set_depth_limit.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_DEPTH_LIMIT 3 "25 March 2017" "PCRE2 10.46"
+.TH PCRE2_SET_DEPTH_LIMIT 3 "25 March 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_glob_escape.3 b/doc/pcre2_set_glob_escape.3
index 7e52e97..89c2ee3 100644
--- a/doc/pcre2_set_glob_escape.3
+++ b/doc/pcre2_set_glob_escape.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_GLOB_ESCAPE 3 "12 July 2017" "PCRE2 10.46"
+.TH PCRE2_SET_GLOB_ESCAPE 3 "12 July 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_glob_separator.3 b/doc/pcre2_set_glob_separator.3
index 36ebdb9..14e8e2f 100644
--- a/doc/pcre2_set_glob_separator.3
+++ b/doc/pcre2_set_glob_separator.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_GLOB_SEPARATOR 3 "17 June 2018" "PCRE2 10.46"
+.TH PCRE2_SET_GLOB_SEPARATOR 3 "17 June 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_heap_limit.3 b/doc/pcre2_set_heap_limit.3
index f240309..05a69f1 100644
--- a/doc/pcre2_set_heap_limit.3
+++ b/doc/pcre2_set_heap_limit.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_HEAP_LIMIT 3 "17 June 2018" "PCRE2 10.46"
+.TH PCRE2_SET_HEAP_LIMIT 3 "17 June 2018" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_match_limit.3 b/doc/pcre2_set_match_limit.3
index c15172c..d5e34c1 100644
--- a/doc/pcre2_set_match_limit.3
+++ b/doc/pcre2_set_match_limit.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_MATCH_LIMIT 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_SET_MATCH_LIMIT 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_max_pattern_compiled_length.3 b/doc/pcre2_set_max_pattern_compiled_length.3
index 2d1e6dd..bfb7fe0 100644
--- a/doc/pcre2_set_max_pattern_compiled_length.3
+++ b/doc/pcre2_set_max_pattern_compiled_length.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_MAX_PATTERN_COMPILED_LENGTH 3 "09 June 2024" "PCRE2 10.46"
+.TH PCRE2_SET_MAX_PATTERN_COMPILED_LENGTH 3 "09 June 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_max_pattern_length.3 b/doc/pcre2_set_max_pattern_length.3
index 7a8b345..7d5aa85 100644
--- a/doc/pcre2_set_max_pattern_length.3
+++ b/doc/pcre2_set_max_pattern_length.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_MAX_PATTERN_LENGTH 3 "05 October 2016" "PCRE2 10.46"
+.TH PCRE2_SET_MAX_PATTERN_LENGTH 3 "05 October 2016" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_max_varlookbehind.3 b/doc/pcre2_set_max_varlookbehind.3
index ce97886..95bb273 100644
--- a/doc/pcre2_set_max_varlookbehind.3
+++ b/doc/pcre2_set_max_varlookbehind.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_NEWLINE 3 "11 August 2023" "PCRE2 10.46"
+.TH PCRE2_SET_NEWLINE 3 "11 August 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_newline.3 b/doc/pcre2_set_newline.3
index 54747b7..40aae44 100644
--- a/doc/pcre2_set_newline.3
+++ b/doc/pcre2_set_newline.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_NEWLINE 3 "19 July 2017" "PCRE2 10.46"
+.TH PCRE2_SET_NEWLINE 3 "19 July 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_offset_limit.3 b/doc/pcre2_set_offset_limit.3
index b9b694f..031b376 100644
--- a/doc/pcre2_set_offset_limit.3
+++ b/doc/pcre2_set_offset_limit.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_OFFSET_LIMIT 3 "22 September 2015" "PCRE2 10.46"
+.TH PCRE2_SET_OFFSET_LIMIT 3 "22 September 2015" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_optimize.3 b/doc/pcre2_set_optimize.3
index 3fc4c9e..499cd62 100644
--- a/doc/pcre2_set_optimize.3
+++ b/doc/pcre2_set_optimize.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_OPTIMIZE 3 "22 September 2024" "PCRE2 10.46"
+.TH PCRE2_SET_OPTIMIZE 3 "22 September 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_parens_nest_limit.3 b/doc/pcre2_set_parens_nest_limit.3
index ccc8542..bc19bf0 100644
--- a/doc/pcre2_set_parens_nest_limit.3
+++ b/doc/pcre2_set_parens_nest_limit.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_PARENS_NEST_LIMIT 3 "25 October 2014" "PCRE2 10.46"
+.TH PCRE2_SET_PARENS_NEST_LIMIT 3 "25 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_recursion_limit.3 b/doc/pcre2_set_recursion_limit.3
index eeaad17..c0bc551 100644
--- a/doc/pcre2_set_recursion_limit.3
+++ b/doc/pcre2_set_recursion_limit.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_RECURSION_LIMIT 3 "19 July 2017" "PCRE2 10.46"
+.TH PCRE2_SET_RECURSION_LIMIT 3 "19 July 2017" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_recursion_memory_management.3 b/doc/pcre2_set_recursion_memory_management.3
index 68bd7ba..392e514 100644
--- a/doc/pcre2_set_recursion_memory_management.3
+++ b/doc/pcre2_set_recursion_memory_management.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_RECURSION_MEMORY_MANAGEMENT 3 "23 January 2023" "PCRE2 10.46"
+.TH PCRE2_SET_RECURSION_MEMORY_MANAGEMENT 3 "23 January 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_substitute_callout.3 b/doc/pcre2_set_substitute_callout.3
index bada5a3..a10359a 100644
--- a/doc/pcre2_set_substitute_callout.3
+++ b/doc/pcre2_set_substitute_callout.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_SUBSTITUTE_CALLOUT 3 "04 October 2024" "PCRE2 10.46"
+.TH PCRE2_SET_SUBSTITUTE_CALLOUT 3 "04 October 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_set_substitute_case_callout.3 b/doc/pcre2_set_substitute_case_callout.3
index 5e54890..ab2f8f4 100644
--- a/doc/pcre2_set_substitute_case_callout.3
+++ b/doc/pcre2_set_substitute_case_callout.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SET_SUBSTITUTE_CASE_CALLOUT 3 "26 December 2024" "PCRE2 10.46"
+.TH PCRE2_SET_SUBSTITUTE_CASE_CALLOUT 3 "26 December 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substitute.3 b/doc/pcre2_substitute.3
index 10ffcc8..b1fbd0c 100644
--- a/doc/pcre2_substitute.3
+++ b/doc/pcre2_substitute.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTITUTE 3 "27 November 2021" "PCRE2 10.46"
+.TH PCRE2_SUBSTITUTE 3 "03 October 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -91,8 +91,9 @@ If PCRE2_SUBSTITUTE_LITERAL is set, PCRE2_SUBSTITUTE_EXTENDED,
PCRE2_SUBSTITUTE_UNKNOWN_UNSET, and PCRE2_SUBSTITUTE_UNSET_EMPTY are ignored.
.P
If PCRE2_SUBSTITUTE_MATCHED is set, \fImatch_data\fP must be non-NULL; its
-contents must be the result of a call to \fBpcre2_match()\fP using the same
-pattern and subject.
+contents must be the result of a call to \fBpcre2_match()\fP (or
+\fBpcre2_jit_match()\fP) using the same pattern, subject pointer, effective
+subject length, start offset, and match options.
.P
The function returns the number of substitutions, which may be zero if there
are no matches. The result may be greater than one only when
diff --git a/doc/pcre2_substring_copy_byname.3 b/doc/pcre2_substring_copy_byname.3
index bffc429..78d05f4 100644
--- a/doc/pcre2_substring_copy_byname.3
+++ b/doc/pcre2_substring_copy_byname.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_COPY_BYNAME 3 "19 December 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_COPY_BYNAME 3 "19 December 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substring_copy_bynumber.3 b/doc/pcre2_substring_copy_bynumber.3
index c1ab9f3..bb0e1ec 100644
--- a/doc/pcre2_substring_copy_bynumber.3
+++ b/doc/pcre2_substring_copy_bynumber.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_COPY_BYNUMBER 3 "13 December 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_COPY_BYNUMBER 3 "13 December 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substring_free.3 b/doc/pcre2_substring_free.3
index 340be6c..47d13ed 100644
--- a/doc/pcre2_substring_free.3
+++ b/doc/pcre2_substring_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_FREE 3 "28 June 2018" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_FREE 3 "26 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -6,8 +6,9 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.sp
.B #include
.PP
-.SM
+.nf
.B void pcre2_substring_free(PCRE2_UCHAR *\fIbuffer\fP);
+.fi
.
.SH DESCRIPTION
.rs
diff --git a/doc/pcre2_substring_get_byname.3 b/doc/pcre2_substring_get_byname.3
index 90e3310..23abdaf 100644
--- a/doc/pcre2_substring_get_byname.3
+++ b/doc/pcre2_substring_get_byname.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_GET_BYNAME 3 "19 December 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_GET_BYNAME 3 "19 December 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substring_get_bynumber.3 b/doc/pcre2_substring_get_bynumber.3
index 8017f2d..ffa9dc6 100644
--- a/doc/pcre2_substring_get_bynumber.3
+++ b/doc/pcre2_substring_get_bynumber.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_GET_BYNUMBER 3 "13 December 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_GET_BYNUMBER 3 "13 December 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substring_length_byname.3 b/doc/pcre2_substring_length_byname.3
index 111fccd..d9cc946 100644
--- a/doc/pcre2_substring_length_byname.3
+++ b/doc/pcre2_substring_length_byname.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_LENGTH_BYNAME 3 "21 October 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_LENGTH_BYNAME 3 "26 September 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -19,9 +19,11 @@ The arguments are:
.sp
\fImatch_data\fP The match data block for the match
\fIname\fP The substring name
- \fIlength\fP Where to return the length
+ \fIlength\fP Where to return the length, or NULL
.sp
-The yield is zero on success, or an error code if the substring is not found.
+The third argument may be NULL if all you want to know is whether or not a
+substring is set. The yield is zero on success, or a negative error code
+otherwise.
.P
There is a complete description of the PCRE2 native API in the
.\" HREF
diff --git a/doc/pcre2_substring_length_bynumber.3 b/doc/pcre2_substring_length_bynumber.3
index bc7c4c1..10e5233 100644
--- a/doc/pcre2_substring_length_bynumber.3
+++ b/doc/pcre2_substring_length_bynumber.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_LENGTH_BYNUMBER 3 "22 December 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_LENGTH_BYNUMBER 3 "22 December 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substring_list_free.3 b/doc/pcre2_substring_list_free.3
index 4e02b69..ee78b4e 100644
--- a/doc/pcre2_substring_list_free.3
+++ b/doc/pcre2_substring_list_free.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_LIST_FREE 3 "02 December 2023" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_LIST_FREE 3 "26 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -6,8 +6,9 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.sp
.B #include
.PP
-.SM
+.nf
.B void pcre2_substring_list_free(PCRE2_UCHAR **\fIlist\fP);
+.fi
.
.SH DESCRIPTION
.rs
diff --git a/doc/pcre2_substring_list_get.3 b/doc/pcre2_substring_list_get.3
index 20cc72c..50b50e4 100644
--- a/doc/pcre2_substring_list_get.3
+++ b/doc/pcre2_substring_list_get.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_LIST_GET 3 "21 October 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_LIST_GET 3 "21 October 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substring_nametable_scan.3 b/doc/pcre2_substring_nametable_scan.3
index 6a9f22a..fb93b2f 100644
--- a/doc/pcre2_substring_nametable_scan.3
+++ b/doc/pcre2_substring_nametable_scan.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_NAMETABLE_SCAN 3 "06 February 2019" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_NAMETABLE_SCAN 3 "06 February 2019" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2_substring_number_from_name.3 b/doc/pcre2_substring_number_from_name.3
index 03b445b..b9be3b5 100644
--- a/doc/pcre2_substring_number_from_name.3
+++ b/doc/pcre2_substring_number_from_name.3
@@ -1,4 +1,4 @@
-.TH PCRE2_SUBSTRING_NUMBER_FROM_NAME 3 "03 November 2014" "PCRE2 10.46"
+.TH PCRE2_SUBSTRING_NUMBER_FROM_NAME 3 "03 November 2014" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
diff --git a/doc/pcre2api.3 b/doc/pcre2api.3
index b09d7ab..a5ea17b 100644
--- a/doc/pcre2api.3
+++ b/doc/pcre2api.3
@@ -1,4 +1,4 @@
-.TH PCRE2API 3 "26 December 2024" "PCRE2 10.46"
+.TH PCRE2API 3 "19 October 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.sp
@@ -105,7 +105,7 @@ document for an overview of all the PCRE2 documentation.
.B " pcre2_compile_context *\fIccontext\fP, PCRE2_SIZE \fIvalue\fP);"
.sp
.B int pcre2_set_max_varlookbehind(pcre2_compile_contest *\fIccontext\fP,
-.B " uint32_t \fIvalue\fP);
+.B " uint32_t \fIvalue\fP);"
.sp
.B int pcre2_set_newline(pcre2_compile_context *\fIccontext\fP,
.B " uint32_t \fIvalue\fP);"
@@ -196,7 +196,7 @@ document for an overview of all the PCRE2 documentation.
.B void pcre2_substring_list_free(PCRE2_UCHAR **\fIlist\fP);
.sp
.B int pcre2_substring_list_get(pcre2_match_data *\fImatch_data\fP,
-.B " PCRE2_UCHAR ***\fIlistptr\fP, PCRE2_SIZE **\fIlengthsptr\fP);
+.B " PCRE2_UCHAR ***\fIlistptr\fP, PCRE2_SIZE **\fIlengthsptr\fP);"
.fi
.
.
@@ -207,7 +207,7 @@ document for an overview of all the PCRE2 documentation.
.B int pcre2_substitute(const pcre2_code *\fIcode\fP, PCRE2_SPTR \fIsubject\fP,
.B " PCRE2_SIZE \fIlength\fP, PCRE2_SIZE \fIstartoffset\fP,"
.B " uint32_t \fIoptions\fP, pcre2_match_data *\fImatch_data\fP,"
-.B " pcre2_match_context *\fImcontext\fP, PCRE2_SPTR \fIreplacementz\fP,"
+.B " pcre2_match_context *\fImcontext\fP, PCRE2_SPTR \fIreplacement\fP,"
.B " PCRE2_SIZE \fIrlength\fP, PCRE2_UCHAR *\fIoutputbuffer\fP,"
.B " PCRE2_SIZE *\fIoutlengthptr\fP);"
.fi
@@ -497,7 +497,7 @@ compiled patterns on disc or elsewhere, and reloading them later.
.P
Finally, there are functions for finding out information about a compiled
pattern (\fBpcre2_pattern_info()\fP) and about the configuration with which
-PCRE2 was built (\fBpcre2_config()\fP).
+PCRE2 was built (\fBpcre2_config()\fP) and that it is using.
.P
Functions with names ending with \fB_free()\fP are used for freeing memory
blocks of various sorts. In all cases, if one of these functions is called with
@@ -831,7 +831,7 @@ can hold, which is effectively unlimited.
.sp
.nf
.B int pcre2_set_max_varlookbehind(pcre2_compile_contest *\fIccontext\fP,
-.B " uint32_t \fIvalue\fP);
+.B " uint32_t \fIvalue\fP);"
.fi
.sp
This sets a maximum length for the number of characters matched by a
@@ -1265,6 +1265,13 @@ The output is a uint32_t integer that gives the default limit for the depth of
nested backtracking in \fBpcre2_match()\fP or the depth of nested recursions,
lookarounds, and atomic groups in \fBpcre2_dfa_match()\fP. Further details are
given with \fBpcre2_set_depth_limit()\fP above.
+.sp
+ PCRE2_CONFIG_EFFECTIVE_LINKSIZE
+.sp
+The output is a uint32_t integer that contains the number of bytes the library
+uses for internal linkage in compiled regular expressions. Its value is derived
+from the value that was provided at build time and that is described below by
+PCRE2_CONFIG_LINKSIZE.
.sp
PCRE2_CONFIG_HEAPLIMIT
.sp
@@ -1289,7 +1296,7 @@ documentation for more details.
.sp
PCRE2_CONFIG_JITTARGET
.sp
-The \fIwhere\fP argument should point to a buffer that is at least 48 code
+The \fIwhere\fP argument should point to a buffer that is at least 64 code
units long. (The exact length required can be found by calling
\fBpcre2_config()\fP with \fBwhere\fP set to NULL.) The buffer is filled with a
string that contains the name of the architecture for which the JIT compiler is
@@ -1300,13 +1307,14 @@ for the terminating zero.
.sp
PCRE2_CONFIG_LINKSIZE
.sp
-The output is a uint32_t integer that contains the number of bytes used for
-internal linkage in compiled regular expressions. When PCRE2 is configured, the
-value can be set to 2, 3, or 4, with the default being 2. This is the value
-that is returned by \fBpcre2_config()\fP. However, when the 16-bit library is
-compiled, a value of 3 is rounded up to 4, and when the 32-bit library is
-compiled, internal linkages always use 4 bytes, so the configured value is not
-relevant.
+The output is a uint32_t integer that contains the number of bytes the library
+was instructed to use for internal linkage in compiled regular expressions.
+When PCRE2 is configured, the value can be set to 2, 3, or 4, with the default
+being 2 for most libraries.
+.P
+The actual number of bytes used depends on the size of the code units that the
+library supports and can be higher. See PCRE2_CONFIG_EFFECTIVE_LINKSIZE above
+for details.
.P
The default value of 2 for the 8-bit and 16-bit libraries is sufficient for all
but the most massive patterns, since it allows the size of the compiled pattern
@@ -1584,8 +1592,7 @@ case the following character).
(3) \ex matches a lower case "x" character unless it is followed by two
hexadecimal digits, in which case the hexadecimal number defines the code point
to match. By default, as in Perl, a hexadecimal number is always expected after
-\ex, but it may have zero, one, or two digits (so, for example, \exz matches a
-binary zero character followed by z).
+\ex, but it may have one or two digits.
.P
ECMAscript 6 added additional functionality to \eu. This can be accessed using
the PCRE2_EXTRA_ALT_BSUX extra option (see "Extra compile options"
@@ -1628,7 +1635,7 @@ option is set, normal backslash processing is applied to verb names and only an
unescaped closing parenthesis terminates the name. A closing parenthesis can be
included in a name either as \e) or between \eQ and \eE. If the PCRE2_EXTENDED
or PCRE2_EXTENDED_MORE option is set with PCRE2_ALT_VERBNAMES, unescaped
-whitespace in verb names is skipped and #-comments are recognized, exactly as
+white space in verb names is skipped and #-comments are recognized, exactly as
in the rest of the pattern.
.sp
PCRE2_AUTO_CALLOUT
@@ -1926,8 +1933,9 @@ instead.)
.sp
This is an option whose main effect is at matching time. It does not change
what \fBpcre2_compile()\fP generates, but it does affect the output of the JIT
-compiler. Setting this option is equivalent to calling \fBpcre2_set_optimize()\fP
-with the \fIdirective\fP parameter set to PCRE2_START_OPTIMIZE_OFF.
+compiler. Setting this option is equivalent to calling
+\fBpcre2_set_optimize()\fP with the \fIdirective\fP parameter set to
+PCRE2_START_OPTIMIZE_OFF.
.P
There are a number of optimizations that may occur at the start of a match, in
order to speed up the process. For example, if it is known that an unanchored
@@ -2012,9 +2020,9 @@ only if PCRE2 has been compiled with Unicode support (which is the default).
.P
The PCRE2_EXTRA_CASELESS_RESTRICT option (see above) restricts caseless
matching such that ASCII characters match only ASCII characters and non-ASCII
-characters match only non-ASCII characters. The PCRE2_EXTRA_TURKISH_CASING option
-(see above) alters the matching of the 'i' characters to follow their behaviour
-in Turkish and Azeri languages. For further details on
+characters match only non-ASCII characters. The PCRE2_EXTRA_TURKISH_CASING
+option (see above) alters the matching of the 'i' characters to follow their
+behaviour in Turkish and Azeri languages. For further details on
PCRE2_EXTRA_CASELESS_RESTRICT and PCRE2_EXTRA_TURKISH_CASING, see the
.\" HREF
\fBpcre2unicode\fP
@@ -2958,18 +2966,13 @@ is able to look behind the starting point to discover that it is preceded by a
letter.
.P
Finding all the matches in a subject is tricky when the pattern can match an
-empty string. It is possible to emulate Perl's /g behaviour by first trying the
-match again at the same offset, with the PCRE2_NOTEMPTY_ATSTART and
-PCRE2_ANCHORED options, and then if that fails, advancing the starting offset
-and trying an ordinary match again. There is some code that demonstrates how to
-do this in the
-.\" HREF
-\fBpcre2demo\fP
+empty string. PCRE2 includes a helper API to assist with this; see the
+section entitled "Iterating over all matches"
+.\" HTML
+.\"
+below
.\"
-sample program. In the most general case, you have to check to see if the
-newline convention recognizes CRLF as a newline, and if so, and the current
-character is CR followed by LF, advance the starting offset by two characters
-instead of one.
+for details.
.P
If a non-zero starting offset is passed when the pattern is anchored, a single
attempt to match at the given offset is made. This can only succeed if the
@@ -2999,7 +3002,7 @@ PCRE2_DISABLE_RECURSELOOP_CHECK is ignored by JIT, but apart from PCRE2_NO_JIT
.sp
The PCRE2_ANCHORED option limits \fBpcre2_match()\fP to matching at the first
matching position. If a pattern was compiled with PCRE2_ANCHORED, or turned out
-to be anchored by virtue of its contents, it cannot be made unachored at
+to be anchored by virtue of its contents, it cannot be made unanchored at
matching time. Note that setting the option at match time disables JIT
matching.
.sp
@@ -3162,7 +3165,7 @@ If PCRE2_PARTIAL_HARD is set, it overrides PCRE2_PARTIAL_SOFT. In this case, if
a partial match is found, \fBpcre2_match()\fP immediately returns
PCRE2_ERROR_PARTIAL, without considering any other alternatives. In other
words, when PCRE2_PARTIAL_HARD is set, a partial match is considered to be more
-important that an alternative complete match.
+important than an alternative complete match.
.P
There is a more detailed discussion of partial and multi-segment matching, with
examples, in the
@@ -3285,10 +3288,10 @@ set. For example, if two substrings have been captured, the returned value is
3. If there are no captured substrings, the return value from a successful
match is 1, indicating that just the first pair of offsets has been set.
.P
-If a pattern uses the \eK escape sequence within a positive assertion, the
-reported start of a successful match can be greater than the end of the match.
-For example, if the pattern (?=ab\eK) is matched against "ab", the start and
-end offset values for the match are 2 and 0.
+If a pattern uses the \eK escape sequence within a positive lookahead assertion,
+the reported start of a successful match can be greater than the end of the
+match. For example, if the pattern (?=ab\eK) is matched against "ab", the start
+and end offset values for the match are 2 and 0.
.P
If a capture group is matched repeatedly within a single match operation, it is
the last portion of the subject that it matched that is returned.
@@ -3524,7 +3527,96 @@ returns the number of code units used, excluding the trailing zero. If the
error number is unknown, the negative error code PCRE2_ERROR_BADDATA is
returned. If the buffer is too small, the message is truncated (but still with
a trailing zero), and the negative error code PCRE2_ERROR_NOMEMORY is returned.
-None of the messages are very long; a buffer size of 120 code units is ample.
+None of the messages is very long; a buffer size of 120 code units is ample.
+.
+.
+.\" HTML
+.SH "ITERATING OVER ALL MATCHES"
+.rs
+.sp
+.nf
+.B int pcre2_next_match(pcre2_match_data *\fImatch_data\fP,
+.B " PCRE2_SIZE *\fIpstart_offset\fP, uint32_t *\fIpoptions\fP);"
+.fi
+.P
+A common task for applications is to implement "global" matching behaviour,
+for example, replacing all matches in the subject; splitting the subject on all
+matches; or simply counting the number of matches. The \fBpcre2_next_match()\fP
+function helps with this task by providing the appropriate parameters for the
+next match attempt (available since PCRE2 10.46).
+.P
+First, a match attempt should be made using one of the matching functions
+(\fBpcre2_match()\fP, \fBpcre2_dfa_match()\fP, or \fBpcre2_jit_match()\fP).
+Then, \fBpcre2_next_match()\fP can be called, providing the same
+\fImatch_data\fP parameter.
+.P
+It returns 0 ("false") if there is no need to make a further match attempt, or
+1 ("true") if another match should be attempted. Returning 1 does not imply that
+there is another match, only that another match should be attempted (which may
+return PCRE2_ERROR_NOMATCH).
+.P
+The *\fIpstart_offset\fP and *\fIpoptions\fP are set if the function returns 1.
+The *\fIpstart_offset\fP should be passed to the next match attempt directly,
+and the *\fIpoptions\fP should be passed to the next match attempt by combining
+with the application's match options using OR.
+.P
+There is some code that demonstrates how to do this in the
+.\" HREF
+\fBpcre2demo\fP
+.\"
+sample program. The general pattern is:
+.sp
+.nf
+ uint32_t app_options = ...;
+ uint32_t global_options = 0;
+ PCRE2_SIZE start_offset = 0;
+ while (1)
+ {
+ int rc = pcre2_match(re, subject, subject_len, start_offset,
+ app_options | global_options, match_data,
+ match_context);
+\&
+ if (rc == PCRE2_ERROR_NOMATCH) break; /* no match, and no more attempts */
+ if (rc < 0) { ... exit }
+\&
+ ...handle the match
+\&
+ if (!pcre2_next_match(match_data, &start_offset, &global_options))
+ break; /* no more attempts */
+ }
+.fi
+.P
+The guarantees provided by \fBpcre2_next_match()\fP are that the start_offset
+will advance, so the loop will definitely terminate. The conditions which
+ensure this are that either: (a) pcre2_next_match() returns 0 (false); or
+(b) the returned *\fIpstart_offset\fP is strictly greater than the previous
+start_offset; or (c) if the previous match was a successful match of the empty
+string then the returned *\fIpstart_offset\fP is equal to the previous
+ovector[1], and *\fIpoptions\fP will be set to PCRE2_NOTEMPTY_ATSTART to prevent
+another empty match from being returned.
+.P
+A loop implemented as shown above will always terminate, unless there is a bug
+in PCRE2. As a measure of "defensive programming", applications are encouraged
+to add an assertion or check to break their loop if it does not make progress
+(and report the issue as a bug).
+.P
+If an application does not use the flag PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK, then
+each match is "well-behaved" and satisfies:
+.sp
+ start_offset <= ovector[0] <= ovector[1].
+.sp
+In this case, the matches found by pcre2_match() with pcre2_next_match() will be
+sorted, non-overlapping (possibly touching), and with no duplicates.
+.P
+Otherwise, if PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK is used, then the guarantees are
+considerably weaker. We do not guarantee that the matches will always advance:
+only that the start_offset will. The matches found by pcre2_match() with
+pcre2_next_match() will be a finite sequence (as pcre2_next_match() ensures that
+start_offset advances, so the search will terminate). The matches can however be
+overlapping, can contain duplicates, and (in truly pathological examples) may
+not even be sorted by ovector[0]. Additionally, each match itself can end before
+it starts (ovector[1] < ovector[0]). We recommend that applications do not set
+PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK.
.
.
.\" HTML
@@ -3563,11 +3655,12 @@ substring zero is available. An attempt to extract any other substring gives
the error PCRE2_ERROR_PARTIAL. The next section describes similar functions for
extracting captured substrings by name.
.P
-If a pattern uses the \eK escape sequence within a positive assertion, the
-reported start of a successful match can be greater than the end of the match.
-For example, if the pattern (?=ab\eK) is matched against "ab", the start and
-end offset values for the match are 2 and 0. In this situation, calling these
-functions with a zero substring number extracts a zero-length empty string.
+If a pattern uses the \eK escape sequence within a positive lookahead assertion,
+the reported start of a successful match can be greater than the end of the
+match. For example, if the pattern (?=ab\eK) is matched against "ab", the start
+and end offset values for the match are 2 and 0. In this situation, calling
+these functions with a zero substring number extracts a zero-length empty
+string.
.P
You can find the length in code units of a captured substring without
extracting it by calling \fBpcre2_substring_length_bynumber()\fP. The first
@@ -3626,7 +3719,7 @@ capturing slots, substring number 1 is unset.
.sp
.nf
.B int pcre2_substring_list_get(pcre2_match_data *\fImatch_data\fP,
-.B " PCRE2_UCHAR ***\fIlistptr\fP, PCRE2_SIZE **\fIlengthsptr\fP);
+.B " PCRE2_UCHAR ***\fIlistptr\fP, PCRE2_SIZE **\fIlengthsptr\fP);"
.sp
.B void pcre2_substring_list_free(PCRE2_UCHAR **\fIlist\fP);
.fi
@@ -3756,7 +3849,8 @@ Matches in which a \eK item in a lookahead in the pattern causes the match to
end before it starts are not supported, and give rise to an error return. For
global replacements, matches in which \eK in a lookbehind causes the match to
start earlier than the point that was reached in the previous iteration are
-also not supported.
+also not supported. (These cases are only possible if the pattern was compiled
+with the backwards-compatibility option PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK.)
.P
The first seven arguments of \fBpcre2_substitute()\fP are the same as for
\fBpcre2_match()\fP, except that the partial matching options are not
@@ -3767,7 +3861,8 @@ allocate memory for the compiled code.
.P
If \fImatch_data\fP is not NULL and PCRE2_SUBSTITUTE_MATCHED is not set, the
provided block is used for all calls to \fBpcre2_match()\fP, and its contents
-afterwards are the result of the final call. For global changes, this will
+afterwards are the result of the final call made internally by
+\fBpcre2_substitute()\fP to the matching function. For global changes, this will
always be a no-match error. The contents of the ovector within the match data
block may or may not have been changed.
.P
@@ -3775,23 +3870,45 @@ As well as the usual options for \fBpcre2_match()\fP, a number of additional
options can be set in the \fIoptions\fP argument of \fBpcre2_substitute()\fP.
One such option is PCRE2_SUBSTITUTE_MATCHED. When this is set, an external
\fImatch_data\fP block must be provided, and it must have already been used for
-an external call to \fBpcre2_match()\fP with the same pattern and subject
-arguments. The data in the \fImatch_data\fP block (return code, offset vector)
-is then used for the first substitution instead of calling \fBpcre2_match()\fP
-from within \fBpcre2_substitute()\fP. This allows an application to check for a
-match before choosing to substitute, without having to repeat the match.
+an external call to \fBpcre2_match()\fP (or \fBpcre2_jit_match()\fP) with the
+same pattern, subject pointer, effective subject length, start offset, and match
+option arguments (substitute-specific options can be added to the \fIoptions\fP
+argument). If any of these parameters is changed, \fBpcre2_substitute()\fP
+returns an error. The data in the \fImatch_data\fP block (return code, offset
+vector) is used for the first substitution instead of calling
+\fBpcre2_match()\fP from within \fBpcre2_substitute()\fP. This allows an
+application to check for a match before choosing to substitute, without having
+to repeat the match.
+.P
+If the contents of the subject buffer are mutated in between \fBpcre2_match()\fP
+and a call to \fBpcre2_substitute()\fP with PCRE2_SUBSTITUTE_MATCHED, the
+behaviour is unsafe; in particular, in this case, PCRE2 is unable to ensure that
+the offsets in the ovector point to the start of characters (with UTF-encoded
+input).
.P
The contents of the externally supplied match data block are not changed when
-PCRE2_SUBSTITUTE_MATCHED is set. If PCRE2_SUBSTITUTE_GLOBAL is also set,
-\fBpcre2_match()\fP is called after the first substitution to check for further
-matches, but this is done using an internally obtained match data block, thus
-always leaving the external block unchanged.
+PCRE2_SUBSTITUTE_MATCHED is set, and so the match block is permitted for use in
+another call using PCRE2_SUBSTITUTE_MATCHED. If PCRE2_SUBSTITUTE_GLOBAL is also
+set, \fBpcre2_match()\fP is called after the first substitution to check for
+furthe matches, but this is done using an internally obtained match data block,
+thus always leaving the external block unchanged.
.P
The \fIcode\fP argument is not used for matching before the first substitution
when PCRE2_SUBSTITUTE_MATCHED is set, but it must be provided, even when
PCRE2_SUBSTITUTE_GLOBAL is not set, because it contains information such as the
UTF setting and the number of capturing parentheses in the pattern.
.P
+When using PCRE2_SUBSTITUTE_MATCHED, you should not modify the subject string
+in between the prior call to \fBpcre2_match()\fP and \fBpcre2_substitute()\fP,
+as the substitution assumes that the passed-in ovector is compatible with the
+subject string. Although PCRE2 does verify that the subject is a pointer to the
+same buffer, it cannot in general verify whether the contents of the buffer have
+changed. For example, if the subject buffer is mutated from one valid UTF-8
+string to another valid string, of the same length in code units, the ovector
+offsets are no longer guaranteed to point to the start of a character. Beware
+that with PCRE2_SUBSTITUTE_MATCHED in UTF mode, the subject string is not
+re-scanned for UTF validity when \fBpcre2_substitute()\fP first uses it.
+.P
The default action of \fBpcre2_substitute()\fP is to return a copy of the
subject string with matched substrings replaced. However, if
PCRE2_SUBSTITUTE_REPLACEMENT_ONLY is set, only the replacement substrings are
@@ -3847,10 +3964,11 @@ recognized:
$` insert the substring that precedes the match
$' insert the substring that follows the match
$_ insert the entire input string
+ $+ insert the highest-numbered capture group which matched
$*MARK or ${*MARK} insert a control verb name
.sp
-Either a group number or a group name can be given for \fIn\fP, for example $2 or
-$NAME. Curly brackets are required only if the following character would be
+Either a group number or a group name can be given for \fIn\fP, for example $2
+or $NAME. Curly brackets are required only if the following character would be
interpreted as part of the number or name. The number may be zero to include
the entire matched string. For example, if the pattern a(b)c is matched with
"=abc=" and the replacement string "+$1$0$1+", the result is "=+babcb+=".
@@ -3877,6 +3995,17 @@ Iteration is implemented by advancing the \fIstartoffset\fP value for each
search, which is always passed the entire subject string. If an offset limit is
set in the match context, searching stops when that limit is reached.
.P
+Because global substitutions apply the pattern repeatedly to the subject string,
+and always iterate over non-overlapping matches, the substitutions done by
+\fBpcre2_substitute()\fP do not match and substitute text inside the replacement
+strings themselves (no recursive/iterative substitution). However, applications
+can easily implement other alternative replacement strategies, such as
+iteratively replacing, then matching and replacing on the result. The
+replacement loop inside \fBpcre2_substitute()\fP is simple and can be emulated
+in client code by allocating a buffer, searching for matches in a loop, and
+calling \fBpcre2_substitute()\fP with PCRE2_SUBSTITUTE_REPLACEMENT_ONLY an
+PCRE2_SUBSTITUTE_MATCHED, and without PCRE2_SUBSTITUTE_GLOBAL.
+.P
You can restrict the effect of a global substitution to a portion of the
subject string by setting either or both of \fIstartoffset\fP and an offset
limit. Here is a \fBpcre2test\fP example:
@@ -3921,9 +4050,9 @@ in a pattern, which in Perl has some ambiguities. Details are given in the
.\"
page.
.P
-The Python form \eg, where the angle brackets are part of the syntax and \fIn\fP
-is either a group name or number, is recognized as an altertive way of
-inserting the contents of a group, for example \eg<3>.
+The Python form \eg, where the angle brackets are part of the syntax and
+\fIn\fP is either a group name or number, is recognized as an alternative way
+of inserting the contents of a group, for example \eg<3>.
.P
There are also four escape sequences for forcing the case of inserted letters.
Case forcing applies to all inserted characters, including those from capture
@@ -4018,7 +4147,7 @@ replacement string, with more particular errors being PCRE2_ERROR_BADREPESCAPE
not found), PCRE2_ERROR_BADSUBSTITUTION (syntax error in extended group
substitution), and PCRE2_ERROR_BADSUBSPATTERN (the pattern match ended before
it started or the match started earlier than the current position in the
-subject, which can happen if \eK is used in an assertion).
+subject, which can happen if \eK is used in a lookaround assertion).
.P
As for all PCRE2 errors, a text message that describes the error can be
obtained by calling the \fBpcre2_get_error_message()\fP function (see
@@ -4039,7 +4168,7 @@ above).
.B " void *\fIcallout_data\fP);"
.fi
.sp
-The \fBpcre2_set_substitution_callout()\fP function can be used to specify a
+The \fBpcre2_set_substitute_callout()\fP function can be used to specify a
callout function for \fBpcre2_substitute()\fP. This information is passed in
a match context. The callout function is called after each substitution has
been processed, but it can cause the replacement not to happen.
@@ -4107,7 +4236,7 @@ matches so far.
.B " void *\fIcallout_data\fP);"
.fi
.sp
-The \fBpcre2_set_substitution_case_callout()\fP function can be used to specify
+The \fBpcre2_set_substitute_case_callout()\fP function can be used to specify
a callout function for \fBpcre2_substitute()\fP to use when performing case
transformations. This does not affect any case insensitivity behaviour when
performing a match, but only the user-visible transformations performed when
@@ -4477,6 +4606,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 26 December 2024
+Last updated: 19 October 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2build.3 b/doc/pcre2build.3
index 787e02d..e7af55a 100644
--- a/doc/pcre2build.3
+++ b/doc/pcre2build.3
@@ -1,4 +1,4 @@
-.TH PCRE2BUILD 3 "16 April 2024" "PCRE2 10.46"
+.TH PCRE2BUILD 3 "17 October 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.
@@ -7,9 +7,9 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.rs
.sp
PCRE2 is distributed with a \fBconfigure\fP script that can be used to build
-the library in Unix-like environments using the applications known as
-Autotools. Also in the distribution are files to support building using
-\fBCMake\fP instead of \fBconfigure\fP. The text file
+the library in Unix-like environments using the Autotools applications. Also in
+the distribution are files to support building using \fBCMake\fP instead of
+\fBconfigure\fP. The text file
.\" HTML
.\"
\fBREADME\fP
@@ -134,8 +134,8 @@ UTF support allows the libraries to process character code points up to
0x10ffff in the strings that they handle. Unicode support also gives access to
the Unicode properties of characters, using pattern escapes such as \eP, \ep,
and \eX. Only the general category properties such as \fILu\fP and \fINd\fP,
-script names, and some bi-directional properties are supported. Details are
-given in the
+script names, and some bi-directional and binary properties are supported.
+Details are given in the
.\" HREF
\fBpcre2pattern\fP
.\"
@@ -152,8 +152,8 @@ request this by starting with (*UCP).
.sp
The \eC escape sequence, which matches a single code unit, even in a UTF mode,
can cause unpredictable behaviour because it may leave the current matching
-point in the middle of a multi-code-unit character. The application can lock it
-out by setting the PCRE2_NEVER_BACKSLASH_C option when calling
+point in the middle of a multi-code-unit character. The application can lock
+it out by setting the PCRE2_NEVER_BACKSLASH_C option when calling
\fBpcre2_compile()\fP. There is also a build-time option
.sp
--enable-never-backslash-C
@@ -395,13 +395,19 @@ most computer operating systems. PCRE2 can, however, be compiled to run in an
.sp
--enable-ebcdic --disable-unicode
.sp
-to the \fBconfigure\fP command. This setting implies
---enable-rebuild-chartables. You should only use it if you know that you are in
-an EBCDIC environment (for example, an IBM mainframe operating system).
+to the \fBconfigure\fP command. You should only use it if you know that you are
+in an EBCDIC environment (for example, an IBM mainframe operating system).
.P
-It is not possible to support both EBCDIC and UTF-8 codes in the same version
-of the library. Consequently, --enable-unicode and --enable-ebcdic are mutually
-exclusive.
+This setting implies --enable-rebuild-chartables, in order to ensure that you
+have the correct default character tables for your system's codepage. There is
+an exception when you set --enable-ebcdic-ignoring-compiler (see below), which
+allows using a default set of EBCDIC 1047 character tables rather than forcing
+use of --enable-rebuild-chartables.
+.P
+It is not supported to enable both EBCDIC input and either ASCII or UTF-8/16/32
+in the same build of the library. When PCRE2 is built with EBCDIC support, it
+always operates in EBCDIC, and consequently --enable-unicode and --enable-ebcdic
+are mutually exclusive.
.P
The EBCDIC character that corresponds to an ASCII LF is assumed to have the
value 0x15 by default. However, in some EBCDIC environments, 0x25 is used. In
@@ -409,14 +415,25 @@ such an environment you should use
.sp
--enable-ebcdic-nl25
.sp
-as well as, or instead of, --enable-ebcdic. The EBCDIC character for CR has the
-same value as in ASCII, namely, 0x0d. Whichever of 0x15 and 0x25 is \fInot\fP
-chosen as LF is made to correspond to the Unicode NEL character (which, in
-Unicode, is 0x85).
+(which implies --enable-ebcdic). The EBCDIC character for CR has the same value
+as in ASCII, namely, 0x0d. Whichever of 0x15 and 0x25 is \fInot\fP chosen as LF
+is made to correspond to the Unicode NEL character (which, in Unicode, is 0x85).
.P
The options that select newline behaviour, such as --enable-newline-is-cr,
and equivalent run-time options, refer to these character values in an EBCDIC
environment.
+.P
+On systems requiring an EBCDIC build of PCRE2, the compiler should be set to use
+the correct codepage, so that C character literals such as 'z' use the correct
+numeric value for whichever EBCDIC codpage is in use. (PCRE2 cannot support
+multiple EBCDIC codepages dynamically.) However, if this not possible, then you
+can use
+.sp
+ --enable-ebcdic-ignoring-compiler
+.sp
+in order to disregard the compiler's codepage, and instead force PCRE2 to use
+numeric constants corresponding to the EBCDIC 1047 codepage instead. This can be
+used to build (or test) EBCDIC support on an ASCII/UTF-8 system such as Linux.
.
.
.SH "PCRE2GREP SUPPORT FOR EXTERNAL SCRIPTS"
@@ -500,7 +517,7 @@ use), some extra configuration may be necessary. The INSTALL file for
If your environment has not been set up so that an appropriate library is
automatically included, you may need to add something like
.sp
- LIBS="-ncurses"
+ LIBS="-lncurses"
.sp
immediately before the \fBconfigure\fP command.
.
@@ -660,6 +677,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 16 April 2024
+Last updated: 17 October 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2callout.3 b/doc/pcre2callout.3
index 3726fbc..6ceefa4 100644
--- a/doc/pcre2callout.3
+++ b/doc/pcre2callout.3
@@ -1,4 +1,4 @@
-.TH PCRE2CALLOUT 3 "19 January 2024" "PCRE2 10.46"
+.TH PCRE2CALLOUT 3 "26 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH SYNOPSIS
@@ -6,7 +6,6 @@ PCRE2 - Perl-compatible regular expressions (revised API)
.sp
.B #include
.PP
-.SM
.nf
.B int (*pcre2_callout)(pcre2_callout_block *, void *);
.sp
@@ -452,6 +451,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 19 January 2024
+Last updated: 26 February 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2compat.3 b/doc/pcre2compat.3
index 131f576..0a6dc1b 100644
--- a/doc/pcre2compat.3
+++ b/doc/pcre2compat.3
@@ -1,4 +1,4 @@
-.TH PCRE2COMPAT 3 "02 October 2024" "PCRE2 10.46"
+.TH PCRE2COMPAT 3 "02 June 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "DIFFERENCES BETWEEN PCRE2 AND PERL"
@@ -34,7 +34,11 @@ any kind of quantifier on non-lookaround assertions.
.P
4. If a braced quantifier such as {1,2} appears where there is nothing to
repeat (for example, at the start of a branch), PCRE2 raises an error whereas
-Perl treats the quantifier characters as literal.
+Perl treats the quantifier characters as literal. When a braced quantifier
+(...){min,max} has min > max, Perl treats it as an item which fails to match
+any portion of the subject (as no number of repetitions can meet the
+condition), and additionally issues a warning when in warning mode. PCRE2 has
+no warning features, so it gives an error in this case.
.P
5. Capture groups that occur inside negative lookaround assertions are counted,
but their entries in the offsets vector are set only when a negative assertion
@@ -186,7 +190,7 @@ variable interpolation, but not general hooks on every match.
.sp
(h) The partial matching facility is PCRE2-specific.
.sp
-(i) The alternative matching function (\fBpcre2_dfa_match()\fP matches in a
+(i) The alternative matching function (\fBpcre2_dfa_match()\fP) matches in a
different way and is not Perl-compatible.
.sp
(j) PCRE2 recognizes some special sequences such as (*CR) or (*NO_JIT) at
@@ -209,7 +213,7 @@ that is not available in Perl.
.P
20. Perl has different limits than PCRE2. See the
.\" HREF
-\fBpcre2limit\fP
+\fBpcre2limits\fP
.\"
documentation for details. Perl went with 5.10 from recursion to iteration
keeping the intermediate matches on the heap, which is ~10% slower but does not
@@ -228,7 +232,7 @@ handled by PCRE2, either by the interpreter or the JIT. An example is
.P
23. Both PCRE2 and Perl error when \ex{ escapes are invalid, but Perl tries to
recover and prints a warning if the problem was that an invalid hexadecimal
-digit was found, since PCRE2 doesn't have warnings it returns an error instead.
+digit was found. Since PCRE2 doesn't have warnings it returns an error instead.
Additionally, Perl accepts \ex{} and generates NUL unlike PCRE2.
.P
24. From release 10.45, PCRE2 gives an error if \ex is not followed by a
@@ -251,6 +255,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 02 October 2024
+Last updated: 02 June 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2convert.3 b/doc/pcre2convert.3
index 8b7a4e6..82448b5 100644
--- a/doc/pcre2convert.3
+++ b/doc/pcre2convert.3
@@ -1,4 +1,4 @@
-.TH PCRE2CONVERT 3 "14 November 2023" "PCRE2 10.46"
+.TH PCRE2CONVERT 3 "14 November 2023" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "EXPERIMENTAL PATTERN CONVERSION FUNCTIONS"
diff --git a/doc/pcre2demo.3 b/doc/pcre2demo.3
index 3f4da3b..fa08962 100644
--- a/doc/pcre2demo.3
+++ b/doc/pcre2demo.3
@@ -1,5 +1,5 @@
-.TH PCRE2DEMO 3 "31 August 2021" "PCRE2 10.46"
-.\"AUTOMATICALLY GENERATED BY PrepareRelease - do not EDIT!
+.TH PCRE2DEMO 3 "24 March 2025" "PCRE2 10.47"
+.\"AUTOMATICALLY GENERATED BY UpdateAlways - do not EDIT!
.SH NAME
PCRE2DEMO - A demonstration C program for PCRE2
.SH "SOURCE CODE"
@@ -101,36 +101,38 @@ PCRE2_SPTR pattern; /* PCRE2_SPTR is a pointer to unsigned code units of */
PCRE2_SPTR subject; /* the appropriate width (in this case, 8 bits). */
PCRE2_SPTR name_table;
-int crlf_is_newline;
int errornumber;
-int find_all;
+int find_all, caseless_match;
int i;
int rc;
-int utf8;
-uint32_t option_bits;
uint32_t namecount;
uint32_t name_entry_size;
-uint32_t newline;
PCRE2_SIZE erroroffset;
PCRE2_SIZE *ovector;
+PCRE2_SIZE ovector_last[2];
PCRE2_SIZE subject_length;
pcre2_match_data *match_data;
/**************************************************************************
-* First, sort out the command line. There is only one possible option at *
-* the moment, "-g" to request repeated matching to find all occurrences, *
-* like Perl's /g option. We set the variable find_all to a non-zero value *
-* if the -g option is present. *
+* First, sort out the command line. Options: *
+* - "-g" to request repeated matching to find all occurrences, *
+* like Perl's /g option. We set the variable find_all to a non-zero *
+* value if the -g option is present. *
+* - "-i" to request caseless matching, like Perl's /i option. We set the *
+* variable caseless_match to PCRE2_CASELESS if the -i option is *
+* present. *
**************************************************************************/
find_all = 0;
+caseless_match = 0;
for (i = 1; i < argc; i++)
{
if (strcmp(argv[i], "-g") == 0) find_all = 1;
+ else if (strcmp(argv[i], "-i") == 0) caseless_match = PCRE2_CASELESS;
else if (argv[i][0] == '-')
{
printf("Unrecognised option %s\en", argv[i]);
@@ -166,7 +168,7 @@ subject_length = (PCRE2_SIZE)strlen((char *)subject);
re = pcre2_compile(
pattern, /* the pattern */
PCRE2_ZERO_TERMINATED, /* indicates pattern is zero-terminated */
- 0, /* default options */
+ caseless_match, /* possibly enable caseless */
&errornumber, /* for error number */
&erroroffset, /* for error offset */
NULL); /* use default compile context */
@@ -246,11 +248,12 @@ if (rc == 0)
printf("ovector was not big enough for all the captured substrings\en");
/* Since release 10.38 PCRE2 has locked out the use of \eK in lookaround
-assertions. However, there is an option to re-enable the old behaviour. If that
-is set, it is possible to run patterns such as /(?=.\eK)/ that use \eK in an
-assertion to set the start of a match later than its end. In this demonstration
-program, we show how to detect this case, but it shouldn't arise because the
-option is never set. */
+assertions. This is the recommended behaviour. However, the option
+PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK allows applications to re-enable the old
+behaviour. If that is set, it is possible to run patterns such as /(?=.\eK)/ that
+use \eK in an assertion to set the start of a match later than its end. In this
+demonstration program, we show how to detect this case, although it cannot arise
+because the option is never set. */
if (ovector[0] > ovector[1])
{
@@ -289,7 +292,9 @@ we have to extract the count of named parentheses from the pattern. */
PCRE2_INFO_NAMECOUNT, /* get the number of named substrings */
&namecount); /* where to put the answer */
-if (namecount == 0) printf("No named substrings\en"); else
+if (namecount == 0)
+ printf("No named substrings\en");
+else
{
PCRE2_SPTR tabptr;
printf("Named substrings\en");
@@ -327,28 +332,8 @@ if (namecount == 0) printf("No named substrings\en"); else
* to search for additional matches in the subject string, in a similar *
* way to the /g option in Perl. This turns out to be trickier than you *
* might think because of the possibility of matching an empty string. *
-* What happens is as follows: *
* *
-* If the previous match was NOT for an empty string, we can just start *
-* the next match at the end of the previous one. *
-* *
-* If the previous match WAS for an empty string, we can't do that, as it *
-* would lead to an infinite loop. Instead, a call of pcre2_match() is *
-* made with the PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set. The *
-* first of these tells PCRE2 that an empty string at the start of the *
-* subject is not a valid match; other possibilities must be tried. The *
-* second flag restricts PCRE2 to one match attempt at the initial string *
-* position. If this match succeeds, an alternative to the empty string *
-* match has been found, and we can print it and proceed round the loop, *
-* advancing by the length of whatever was found. If this match does not *
-* succeed, we still stay in the loop, advancing by just one character. *
-* In UTF-8 mode, which can be set by (*UTF) in the pattern, this may be *
-* more than one byte. *
-* *
-* However, there is a complication concerned with newlines. When the *
-* newline convention is such that CRLF is a valid newline, we must *
-* advance by two characters rather than one. The newline convention can *
-* be set in the regex by (*CR), etc.; if not, we must find the default. *
+* To help with this task, PCRE2 provides the pcre2_next_match() helper. *
*************************************************************************/
if (!find_all) /* Check for -g */
@@ -358,60 +343,21 @@ if (!find_all) /* Check for -g */
return 0; /* Exit the program. */
}
-/* Before running the loop, check for UTF-8 and whether CRLF is a valid newline
-sequence. First, find the options with which the regex was compiled and extract
-the UTF state. */
-
-(void)pcre2_pattern_info(re, PCRE2_INFO_ALLOPTIONS, &option_bits);
-utf8 = (option_bits & PCRE2_UTF) != 0;
-
-/* Now find the newline convention and see whether CRLF is a valid newline
-sequence. */
-
-(void)pcre2_pattern_info(re, PCRE2_INFO_NEWLINE, &newline);
-crlf_is_newline = newline == PCRE2_NEWLINE_ANY ||
- newline == PCRE2_NEWLINE_CRLF ||
- newline == PCRE2_NEWLINE_ANYCRLF;
-
/* Loop for second and subsequent matches */
+ovector_last[0] = ovector[0];
+ovector_last[1] = ovector[1];
+
for (;;)
{
- uint32_t options = 0; /* Normally no options */
- PCRE2_SIZE start_offset = ovector[1]; /* Start at end of previous match */
+ PCRE2_SIZE start_offset;
+ uint32_t options;
- /* If the previous match was for an empty string, we are finished if we are
- at the end of the subject. Otherwise, arrange to run another match at the
- same point to see if a non-empty match can be found. */
+ /* After each successful match, we use pcre2_next_match() to obtain the match
+ parameters for subsequent match attempts. */
- if (ovector[0] == ovector[1])
- {
- if (ovector[0] == subject_length) break;
- options = PCRE2_NOTEMPTY_ATSTART | PCRE2_ANCHORED;
- }
-
- /* If the previous match was not an empty string, there is one tricky case to
- consider. If a pattern contains \eK within a lookbehind assertion at the
- start, the end of the matched string can be at the offset where the match
- started. Without special action, this leads to a loop that keeps on matching
- the same substring. We must detect this case and arrange to move the start on
- by one character. The pcre2_get_startchar() function returns the starting
- offset that was passed to pcre2_match(). */
-
- else
- {
- PCRE2_SIZE startchar = pcre2_get_startchar(match_data);
- if (start_offset <= startchar)
- {
- if (startchar >= subject_length) break; /* Reached end of subject. */
- start_offset = startchar + 1; /* Advance by one character. */
- if (utf8) /* If UTF-8, it may be more */
- { /* than one code unit. */
- for (; start_offset < subject_length; start_offset++)
- if ((subject[start_offset] & 0xc0) != 0x80) break;
- }
- }
- }
+ if (!pcre2_next_match(match_data, &start_offset, &options))
+ break;
/* Run the next matching operation */
@@ -424,38 +370,10 @@ for (;;)
match_data, /* block for storing the result */
NULL); /* use default match context */
- /* This time, a result of NOMATCH isn't an error. If the value in "options"
- is zero, it just means we have found all possible matches, so the loop ends.
- Otherwise, it means we have failed to find a non-empty-string match at a
- point where there was a previous empty-string match. In this case, we do what
- Perl does: advance the matching position by one character, and continue. We
- do this by setting the "end of previous match" offset, because that is picked
- up at the top of the loop as the point at which to start again.
-
- There are two complications: (a) When CRLF is a valid newline sequence, and
- the current position is just before it, advance by an extra byte. (b)
- Otherwise we must ensure that we skip an entire UTF character if we are in
- UTF mode. */
+ /* If this match attempt fails, exit the loop for subsequent matches. */
if (rc == PCRE2_ERROR_NOMATCH)
- {
- if (options == 0) break; /* All matches found */
- ovector[1] = start_offset + 1; /* Advance one code unit */
- if (crlf_is_newline && /* If CRLF is a newline & */
- start_offset < subject_length - 1 && /* we are at CRLF, */
- subject[start_offset] == '\er' &&
- subject[start_offset + 1] == '\en')
- ovector[1] += 1; /* Advance by one more. */
- else if (utf8) /* Otherwise, ensure we */
- { /* advance a whole UTF-8 */
- while (ovector[1] < subject_length) /* character. */
- {
- if ((subject[ovector[1]] & 0xc0) != 0x80) break;
- ovector[1] += 1;
- }
- }
- continue; /* Go round the loop again */
- }
+ break;
/* Other matching errors are not recoverable. */
@@ -467,7 +385,33 @@ for (;;)
return 1;
}
- /* Match succeeded */
+ /* This demonstration program depends on pcre2_next_match() to ensure that the
+ loop for second and subsequent matches does not run forever. However, it would
+ be robust practice for a production application to verify this. The following
+ block of code shows how to do this. This error case is not reachable unless
+ there is a bug in PCRE2.
+
+ Because this program does not set the PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK option,
+ the logic is simple. We verify that either ovector[1] has advanced, or that we
+ have an empty match touching the end of a previous non-empty match. See the
+ API documentation for guidance if your application uses
+ PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK and searches for multiple matches. */
+
+ if (!(ovector[1] > ovector_last[1] ||
+ (ovector[1] == ovector[0] && ovector_last[1] > ovector_last[0] &&
+ ovector[1] == ovector_last[1])))
+ {
+ printf("\e\eK was used in an assertion to yield non-advancing matches.\en");
+ printf("Run abandoned\en");
+ pcre2_match_data_free(match_data);
+ pcre2_code_free(re);
+ return 1;
+ }
+
+ ovector_last[0] = ovector[0];
+ ovector_last[1] = ovector[1];
+
+ /* Match succeeded. */
printf("\enMatch succeeded again at offset %d\en", (int)ovector[0]);
@@ -477,9 +421,11 @@ for (;;)
if (rc == 0)
printf("ovector was not big enough for all the captured substrings\en");
- /* We must guard against patterns such as /(?=.\eK)/ that use \eK in an
- assertion to set the start of a match later than its end. In this
- demonstration program, we just detect this case and give up. */
+ /* We guard against patterns such as /(?=.\eK)/ that use \eK in an assertion to
+ set the start of a match later than its end. As explained above, this case
+ should not occur because this demonstration program does not set the
+ PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK option, however, we do include code showing
+ how to detect it. */
if (ovector[0] > ovector[1])
{
@@ -502,7 +448,9 @@ for (;;)
printf("%2d: %.*s\en", i, (int)substring_length, (char *)substring_start);
}
- if (namecount == 0) printf("No named substrings\en"); else
+ if (namecount == 0)
+ printf("No named substrings\en");
+ else
{
PCRE2_SPTR tabptr = name_table;
printf("Named substrings\en");
@@ -517,6 +465,7 @@ for (;;)
} /* End of loop to find second and subsequent matches */
printf("\en");
+
pcre2_match_data_free(match_data);
pcre2_code_free(re);
return 0;
diff --git a/doc/pcre2grep.1 b/doc/pcre2grep.1
index 95872f1..559d923 100644
--- a/doc/pcre2grep.1
+++ b/doc/pcre2grep.1
@@ -1,4 +1,4 @@
-.TH PCRE2GREP 1 "04 February 2025" "PCRE2 10.46"
+.TH PCRE2GREP 1 "24 January 2025" "PCRE2 10.47"
.SH NAME
pcre2grep - a grep with Perl-compatible regular expressions.
.SH SYNOPSIS
@@ -1022,6 +1022,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 04 February 2025
+Last updated: 24 January 2025
Copyright (c) 1997-2023 University of Cambridge.
.fi
diff --git a/doc/pcre2grep.txt b/doc/pcre2grep.txt
index dd1a38e..55e362a 100644
--- a/doc/pcre2grep.txt
+++ b/doc/pcre2grep.txt
@@ -1099,8 +1099,8 @@ AUTHOR
REVISION
- Last updated: 04 February 2025
+ Last updated: 24 January 2025
Copyright (c) 1997-2023 University of Cambridge.
-PCRE2 10.46 04 February 2025 PCRE2GREP(1)
+PCRE2 10.47 24 January 2025 PCRE2GREP(1)
diff --git a/doc/pcre2jit.3 b/doc/pcre2jit.3
index 009a5ff..620b40c 100644
--- a/doc/pcre2jit.3
+++ b/doc/pcre2jit.3
@@ -1,4 +1,4 @@
-.TH PCRE2JIT 3 "22 August 2024" "PCRE2 10.46"
+.TH PCRE2JIT 3 "22 August 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "PCRE2 JUST-IN-TIME COMPILER SUPPORT"
diff --git a/doc/pcre2limits.3 b/doc/pcre2limits.3
index f51a7a8..b5122ed 100644
--- a/doc/pcre2limits.3
+++ b/doc/pcre2limits.3
@@ -1,4 +1,4 @@
-.TH PCRE2LIMITS 3 "16 August 2023" "PCRE2 10.46"
+.TH PCRE2LIMITS 3 "03 September 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "SIZE AND OTHER LIMITATIONS"
@@ -47,8 +47,13 @@ when PCRE2 is built; if not, the default is set to 250. An application can
change this limit by calling pcre2_set_parens_nest_limit() to set the limit in
a compile context.
.P
-The maximum length of name for a named capture group is 32 code units, and the
-maximum number of such groups is 10000.
+The maximum length of the name for a named capture group as well as the number
+of such groups is configurable at build time. The maximum length for the name
+defaults to
+.\" DEFINE MAX_NAME_SIZE
+128 code units, and the maximum number of such groups to
+.\" DEFINE MAX_NAME_COUNT
+10000.
.P
The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or (*THEN) verb
is 255 code units for the 8-bit library and 65535 code units for the 16-bit and
@@ -76,6 +81,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 16 August 2023
+Last updated: 03 September 2025
Copyright (c) 1997-2023 University of Cambridge.
.fi
diff --git a/doc/pcre2matching.3 b/doc/pcre2matching.3
index 18ee75a..f84b071 100644
--- a/doc/pcre2matching.3
+++ b/doc/pcre2matching.3
@@ -1,4 +1,4 @@
-.TH PCRE2MATCHING 3 "30 August 2024" "PCRE2 10.46"
+.TH PCRE2MATCHING 3 "22 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "PCRE2 MATCHING ALGORITHMS"
@@ -7,7 +7,7 @@ PCRE2 - Perl-compatible regular expressions (revised API)
This document describes the two different algorithms that are available in
PCRE2 for matching a compiled regular expression against a given subject
string. The "standard" algorithm is the one provided by the \fBpcre2_match()\fP
-function. This works in the same as Perl's matching function, and provides a
+function. This works in the same way as Perl's matching function, and provides a
Perl-compatible matching operation. The just-in-time (JIT) optimization that is
described in the
.\" HREF
@@ -223,6 +223,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 30 August 2024
+Last updated: 22 February 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2partial.3 b/doc/pcre2partial.3
index 0674f71..42a5c36 100644
--- a/doc/pcre2partial.3
+++ b/doc/pcre2partial.3
@@ -1,4 +1,4 @@
-.TH PCRE2PARTIAL 3 "27 November 2024" "PCRE2 10.46"
+.TH PCRE2PARTIAL 3 "27 November 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "PARTIAL MATCHING IN PCRE2"
diff --git a/doc/pcre2pattern.3 b/doc/pcre2pattern.3
index 242ba7a..0ad38db 100644
--- a/doc/pcre2pattern.3
+++ b/doc/pcre2pattern.3
@@ -1,4 +1,4 @@
-.TH PCRE2PATTERN 3 "27 November 2024" "PCRE2 10.46"
+.TH PCRE2PATTERN 3 "03 September 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "PCRE2 REGULAR EXPRESSION DETAILS"
@@ -150,10 +150,11 @@ documentation.
.rs
.sp
If a pattern starts with (*NO_DOTSTAR_ANCHOR), it has the same effect as
-setting the PCRE2_NO_DOTSTAR_ANCHOR option, or calling \fBpcre2_set_optimize()\fP
-with a PCRE2_DOTSTAR_ANCHOR_OFF directive. This disables optimizations that
-apply to patterns whose top-level branches all start with .* (match any number
-of arbitrary characters). For more details, see the
+setting the PCRE2_NO_DOTSTAR_ANCHOR option, or
+calling \fBpcre2_set_optimize()\fP with a PCRE2_DOTSTAR_ANCHOR_OFF directive.
+This disables optimizations that apply to patterns whose top-level branches
+all start with .* (match any number of arbitrary characters). For more details,
+see the
.\" HREF
\fBpcre2api\fP
.\"
@@ -1360,7 +1361,7 @@ it matches any character except one that signifies the end of a line.
.P
When \eN is followed by an opening brace it has a different meaning. See the
section entitled
-.\" HTML
+.\" HTML
.\"
"Non-printing characters"
.\"
@@ -2015,8 +2016,9 @@ the naming of capture groups. This feature was not added to Perl until release
using the Python syntax. PCRE2 supports both the Perl and the Python syntax.
.P
In PCRE2, a capture group can be named in one of three ways: (?...) or
-(?'name'...) as in Perl, or (?P...) as in Python. Names may be up to 128
-code units long. When PCRE2_UTF is not set, they may contain only ASCII
+(?'name'...) as in Perl, or (?P...) as in Python. Names may be up to
+.\" DEFINE MAX_NAME_SIZE
+128 code units long. When PCRE2_UTF is not set, they may contain only ASCII
alphanumeric characters and underscores, but must start with a non-digit. When
PCRE2_UTF is set, the syntax of group names is extended to allow any Unicode
letter or Unicode decimal digit. In other words, group names must match one of
@@ -3184,8 +3186,7 @@ For example:
(?(VERSION>=10.4)yes|no)
.sp
This pattern matches "yes" if the PCRE2 version is greater or equal to 10.4, or
-"no" otherwise. The fractional part of the version number may not contain more
-than two digits.
+"no" otherwise. The fractional part of the version number could be ommited.
.
.
.SS "Assertion conditions"
@@ -3442,7 +3443,7 @@ later versions (I tried 5.024) it now works.
.
.
.\" HTML
-.SH "GROUPS AS SUBROUTINES"
+.SS "Groups as subroutines"
.rs
.sp
If the syntax for a recursive group call (either by number or by name) is used
@@ -3495,8 +3496,51 @@ in groups when called as subroutines is described in the section entitled
below.
.
.
+.SS "Recursion and subroutines with returned capture groups"
+.rs
+.sp
+Since PCRE2 10.46, recursion and subroutine calls may also specify a list of
+capture groups to return. This is a PCRE2 syntax extension not supported by
+Perl. The pattern matching recurses into the referenced expression as described
+above, however, when the recursion returns to the calling expression the
+subgroups captured during the recursion can be retained when the calling
+expression's context is restored.
+.P
+When used as a subroutine, this allows the subroutine's capture groups to
+be used as return values.
+.P
+Only the specific capture groups listed by the caller will be retained, using
+the following syntax:
+.sp
+ (?R(grouplist)) recurse whole pattern, returning capture groups
+ (?n(grouplist)) )
+ (?+n(grouplist)) )
+ (?-n(grouplist)) ) call subroutine, returning capture groups
+ (?&name(grouplist)) )
+ (?P>name(grouplist)) )
+.P
+The list of capture groups "grouplist" is a comma-separated list of (absolute
+or relative) group numbers, and group names enclosed in single quotes or angle
+brackets.
+.P
+Here is an example which first uses the DEFINE condition to create a re-usable
+routine for matching a weekday, then calls that subroutine and retains the
+groups it captures for use later:
+.sp
+ (?x: # ignore whitespace for clarity
+ # Define the routine "weekendday" which matches Saturday or
+ # Sunday, and returns the Sat/Sun prefix as \ek.
+ (?(DEFINE) (?
+ (?|(?Sat)urday|(?Sun)day) ) )
+ # Call the routine. Matches "Saturday,Sat" or "Sunday,Sun".
+ (?&weekendday()),\ek )
+.P
+This feature is not available using the Oniguruma syntax \eg<...> or \eg'...'
+below.
+.
+.
.\" HTML
-.SH "ONIGURUMA SUBROUTINE SYNTAX"
+.SS "Oniguruma subroutine syntax"
.rs
.sp
For compatibility with Oniguruma, the non-Perl syntax \eg followed by a name or
@@ -3618,7 +3662,7 @@ are faulted.
.P
A closing parenthesis can be included in a name either as \e) or between \eQ
and \eE. In addition to backslash processing, if the PCRE2_EXTENDED or
-PCRE2_EXTENDED_MORE option is also set, unescaped whitespace in verb names is
+PCRE2_EXTENDED_MORE option is also set, unescaped white space in verb names is
skipped, and #-comments are recognized, exactly as in the rest of the pattern.
PCRE2_EXTENDED and PCRE2_EXTENDED_MORE do not affect verb names unless
PCRE2_ALT_VERBNAMES is also set.
@@ -4186,6 +4230,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 27 November 2024
+Last updated: 03 September 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2perform.3 b/doc/pcre2perform.3
index 2d3e33b..f419559 100644
--- a/doc/pcre2perform.3
+++ b/doc/pcre2perform.3
@@ -1,4 +1,4 @@
-.TH PCRE2PERFORM 3 "06 December 2022" "PCRE2 10.46"
+.TH PCRE2PERFORM 3 "06 December 2022" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "PCRE2 PERFORMANCE"
diff --git a/doc/pcre2posix.3 b/doc/pcre2posix.3
index 4eceda1..359f017 100644
--- a/doc/pcre2posix.3
+++ b/doc/pcre2posix.3
@@ -1,4 +1,4 @@
-.TH PCRE2POSIX 3 "27 November 2024" "PCRE2 10.46"
+.TH PCRE2POSIX 3 "27 November 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "SYNOPSIS"
diff --git a/doc/pcre2sample.3 b/doc/pcre2sample.3
index a55b795..505c3a7 100644
--- a/doc/pcre2sample.3
+++ b/doc/pcre2sample.3
@@ -1,4 +1,4 @@
-.TH PCRE2SAMPLE 3 "14 November 2023" "PCRE2 10.46"
+.TH PCRE2SAMPLE 3 "28 February 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "PCRE2 SAMPLE PROGRAM"
@@ -11,7 +11,7 @@ distribution. A listing of this program is given in the
\fBpcre2demo\fP
.\"
documentation. If you do not have a copy of the PCRE2 distribution, you can
-save this listing to re-create the contents of \fIpcre2demo.c\fP.
+save this listing to recreate the contents of \fIpcre2demo.c\fP.
.P
The demonstration program compiles the regular expression that is its
first argument, and matches it against the subject string in its second
@@ -50,6 +50,7 @@ this:
.sp
./pcre2demo 'cat|dog' 'the cat sat on the mat'
./pcre2demo -g 'cat|dog' 'the dog sat on the cat'
+ ./pcre2demo -i 'cat' 'the dog sat on the CAT'
.sp
Note that there is a much more comprehensive test program, called
.\" HREF
@@ -94,6 +95,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 14 November 2023
+Last updated: 28 February 2025
Copyright (c) 1997-2016 University of Cambridge.
.fi
diff --git a/doc/pcre2serialize.3 b/doc/pcre2serialize.3
index 4de8e30..15aa8fa 100644
--- a/doc/pcre2serialize.3
+++ b/doc/pcre2serialize.3
@@ -1,4 +1,4 @@
-.TH PCRE2SERIALIZE 3 "19 January 2024" "PCRE2 10.46"
+.TH PCRE2SERIALIZE 3 "19 January 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "SAVING AND RE-USING PRECOMPILED PCRE2 PATTERNS"
diff --git a/doc/pcre2syntax.3 b/doc/pcre2syntax.3
index 0fb37d9..d1a611b 100644
--- a/doc/pcre2syntax.3
+++ b/doc/pcre2syntax.3
@@ -1,4 +1,4 @@
-.TH PCRE2SYNTAX 3 "27 November 2024" "PCRE2 10.46"
+.TH PCRE2SYNTAX 3 "14 October 2025" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "PCRE2 REGULAR EXPRESSION SYNTAX SUMMARY"
@@ -303,7 +303,7 @@ used, allowing nested character classes, combined using set operators.
.rs
.sp
(?[...]) Perl extended character class
- (?[\ep{Thai} & \ep{Nd}]) operators; whitespace ignored
+ (?[\ep{Thai} & \ep{Nd}]) operators; white space ignored
(?[(x - y) & z]) parentheses for grouping
.sp
(?[ [^3] & \ep{Nd} ]) [...] is a nested ordinary class
@@ -543,14 +543,14 @@ This feature is not Perl-compatible.
(*scan_substring:(grouplist)...) scan captured substring
(*scs:(grouplist)...) scan captured substring
.sp
-The comma-separated list may identify groups in any of the following ways:
+The comma-separated list "grouplist" may identify groups in any of the
+following ways:
.sp
n absolute reference
+n relative reference
-n relative reference
name
'name' name
-.sp
.
.
.SH "SCRIPT RUNS"
@@ -597,6 +597,28 @@ The comma-separated list may identify groups in any of the following ways:
\eg'+n' call subroutine by relative number (PCRE2 extension)
\eg<-n> call subroutine by relative number (PCRE2 extension)
\eg'-n' call subroutine by relative number (PCRE2 extension)
+.sp
+The variants using parentheses (?...) may also specify a list of capture groups
+to return, which shall be retained in the calling subexpression if set during
+the recursion (this feature is not supported by Perl).
+.sp
+ (?R(grouplist)) recurse whole pattern, returning capture groups
+ (PCRE2 extension)
+ (?n(grouplist)) )
+ (?+n(grouplist)) ) call subroutine, returning capture groups
+ (?-n(grouplist)) ) (PCRE2 extension)
+ (?&name(grouplist)) )
+ (?P>name(grouplist)) )
+.sp
+The comma-separated list "grouplist" uses the same syntax as
+(*scan_substring:(grouplist)...), and may identify groups in any of the
+following ways:
+.sp
+ n absolute reference
+ +n relative reference
+ -n relative reference
+ name
+ 'name' name
.
.
.SH "CONDITIONAL PATTERNS"
@@ -605,22 +627,25 @@ The comma-separated list may identify groups in any of the following ways:
(?(condition)yes-pattern)
(?(condition)yes-pattern|no-pattern)
.sp
- (?(n) absolute reference condition
- (?(+n) relative reference condition (PCRE2 extension)
- (?(-n) relative reference condition (PCRE2 extension)
- (?() named reference condition (Perl)
- (?('name') named reference condition (Perl)
- (?(name) named reference condition (PCRE2, deprecated)
- (?(R) overall recursion condition
- (?(Rn) specific numbered group recursion condition
- (?(R&name) specific named group recursion condition
- (?(DEFINE) define groups for reference
- (?(VERSION[>]=n.m) test PCRE2 version
- (?(assert) assertion condition
+ (?(n) absolute reference condition
+ (?(+n) relative reference condition (PCRE2 extension)
+ (?(-n) relative reference condition (PCRE2 extension)
+ (?() named reference condition (Perl)
+ (?('name') named reference condition (Perl)
+ (?(name) named reference condition (PCRE2, deprecated)
+ (?(R) overall recursion condition
+ (?(Rn) specific numbered group recursion condition
+ (?(R&name) specific named group recursion condition
+ (?(DEFINE) define groups for reference
+ (?(VERSION[>]=n[.m]) test PCRE2 version
+ (?(assert) assertion condition
.sp
Note the ambiguity of (?(R) and (?(Rn) which might be named reference
conditions or recursion tests. Such a condition is interpreted as a reference
condition if the relevant named group exists.
+.sp
+The parts within brackets for the VERSION conditional syntax could be ommited.
+The fractional part of the version number defaults to 0 in that case.
.
.
.SH "BACKTRACKING CONTROL"
@@ -678,6 +703,7 @@ special character is the dollar character in one of the following forms:
$` insert the substring that precedes the match
$' insert the substring that follows the match
$_ insert the entire input string
+ $+ insert the highest-numbered capture group which matched
$*MARK or ${*MARK} insert a control verb name
.sp
For ${n}, n can be a name or a number. If PCRE2_SUBSTITUTE_EXTENDED is set,
@@ -686,16 +712,16 @@ there is additional interpretation:
1. Backslash is an escape character, and the forms described in "ESCAPED
CHARACTERS" above are recognized. Also:
.sp
- \eQ...\eE can be used to suppress interpretation
- \el force the next character to lower case
- \eu force the next character to upper case
- \eL force subsequent characters to lower case
- \eU force subsequent characters to upper case
- \eu\eL force next character to upper case, then all lower
- \el\eU force next character to lower case, then all upper
- \eE end \eL or \eU case forcing
- \eb backspace character (note: as in character class in pattern)
- \ev vertical tab character (note: not the same as in a pattern)
+ \eQ...\eE can be used to suppress interpretation
+ \el force the next character to lower case
+ \eu force the next character to upper case
+ \eL force subsequent characters to lower case
+ \eU force subsequent characters to upper case
+ \eu\eL force next character to upper case, then all lower
+ \el\eU force next character to lower case, then all upper
+ \eE end \eL or \eU case forcing
+ \eb backspace character (note: as in character class in pattern)
+ \ev vertical tab character (note: not the same as in a pattern)
.sp
2. The Python form \eg, where the angle brackets are part of the syntax and
\fIn\fP is either a group name or a number, is recognized as an alternative way
@@ -731,6 +757,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 27 November 2024
+Last updated: 14 October 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2test.1 b/doc/pcre2test.1
index a84b62a..7dbc731 100644
--- a/doc/pcre2test.1
+++ b/doc/pcre2test.1
@@ -1,4 +1,4 @@
-.TH PCRE2TEST 1 "26 December 2024" "PCRE2 10.46"
+.TH PCRE2TEST 1 "12 October 2025" "PCRE2 10.47"
.SH NAME
pcre2test - a program for testing Perl-compatible regular expressions.
.SH SYNOPSIS
@@ -143,10 +143,6 @@ Output information about a specific build-time option, then exit. This
functionality is intended for use in scripts such as \fBRunTest\fP. The
following options output the value and set the exit code as indicated:
.sp
- ebcdic-nl the code for LF (= NL) in an EBCDIC environment:
- either 0x15 or 0x25
- 0 if used in an ASCII/Unicode environment
- exit code is always 0
linksize the configured internal link size (2, 3, or 4)
exit code is set to the link size
newline the default newline setting:
@@ -161,6 +157,10 @@ to the same value:
.sp
backslash-C \eC is supported (not locked out)
ebcdic compiled for an EBCDIC environment
+ ebcdic-io if PCRE2 is compiled for EBCDIC, whether pcre2test's input and
+ output is EBCDIC or ASCII
+ ebcdic-nl25 if PCRE2 is compiled for EBCDIC, whether NL (= LF) is 0x25
+ (otherwise it is 0x15, the default)
jit just-in-time support is available
pcre2-16 the 16-bit library was built
pcre2-32 the 32-bit library was built
@@ -179,6 +179,11 @@ information. It returns one of the following values:
.sp
If an unknown option is given, an error message is output; the exit code is 0.
.TP 10
+\fB--colo[u]r[=]\fP
+By default, the output is coloured if the output file is a terminal (\fBauto\fP).
+Force or suppress output of ANSI colour escapes with \fBalways\fP and \fBnever\fP
+respectively.
+.TP 10
\fB-d\fP
Behave as if each pattern has the \fBdebug\fP modifier; the internal
form and information about the compiled pattern is output after compilation;
@@ -189,6 +194,10 @@ Behave as if each subject line has the \fBdfa\fP modifier; matching is done
using the \fBpcre2_dfa_match()\fP function instead of the default
\fBpcre2_match()\fP.
.TP 10
+\fB-E\fP
+Run in "preprocess only" mode (similar to "gcc -E"). The "#if ... #endif"
+commands are processed, and all other lines are printed verbatim.
+.TP 10
\fB-error\fP \fInumber[,number,...]\fP
Call \fBpcre2_get_error_message()\fP for each of the error numbers in the
comma-separated list, display the resulting messages on the standard output,
@@ -232,6 +241,11 @@ List scripts: write a list of recognized Unicode script names to the standard
output, then exit with zero exit code. All other options are ignored. If both
-C and any -Lx options are present, whichever is first is recognized.
.TP 10
+\fB-malloc\fP
+Exercise malloc() failures, by first counting the number of calls made to malloc
+during pattern compilation and matching, then re-running the compilation and
+matching that many times, exercising a failure of each malloc() call.
+.TP 10
\fB-pattern\fP \fImodifier-list\fP
Behave as if each pattern line contains the given modifiers.
.TP 10
@@ -261,6 +275,10 @@ compile phase.
These behave like \fB-t\fP and \fB-tm\fP, but in addition, at the end of a run,
the total times for all compiles and matches are output.
.TP 10
+\fB-unittest\fP
+Run a fixed set of additional tests of the PCRE2 API which are not driven by
+the test input files, and then exit.
+.TP 10
\fB-version\fP
Output the PCRE2 version number and then exit.
.
@@ -420,13 +438,24 @@ below.
.sp
This command sets a default modifier list that applies to all subsequent
subject lines. Modifiers on a subject line can change these settings.
+.sp
+ #if CONDITION
+ ...
+ #endif
+.sp
+If CONDITION is true, then the command is printed, and its contents are
+processed as normal, including printing the commandlines to the output. If
+CONDITION is false, then all lines between the "#if" and "#endif" are skipped
+and not printed. The CONDITION can be any of the conditions which are tested by
+the "-C" commandline option and which set pcre2test's exit code to a boolean
+value. The CONDITION may also be preceded by "!".
.
.
.SH "MODIFIER SYNTAX"
.rs
.sp
Modifier lists are used with both pattern and subject lines. Items in a list
-are separated by commas followed by optional white space. Trailing whitespace
+are separated by commas followed by optional white space. Trailing white space
in a modifier list is ignored. Some modifiers may be given for both patterns
and subject lines, whereas others are valid only for one or the other. Each
modifier has a long name, for example "anchored", and some of them must be
@@ -550,7 +579,7 @@ the start of a modifier list. For example:
.sp
abc\e=notbol,notempty
.sp
-If the subject string is empty and \e= is followed by whitespace, the line is
+If the subject string is empty and \e= is followed by white space, the line is
treated as a comment line, and is not used for matching. For example:
.sp
\e= This is a comment.
@@ -1126,7 +1155,8 @@ process.
heapframes_size show match data heapframes size
jitstack= set size of JIT stack
mark show mark values
- replace= specify a replacement string
+ null_substitute_match_data substitute with NULL match data
+ replace= specify a replacement string
startchar show starting character when relevant
substitute_callout use substitution callouts
substitute_case_callout use substitution case callouts
@@ -1308,11 +1338,12 @@ pattern, but can be overridden by modifiers on the subject.
null_context match with a NULL context
null_replacement substitute with NULL replacement
null_subject match with NULL subject
+ null_substitute_match_data substitute with NULL match data
offset= set starting offset
offset_limit= set offset limit
ovector= set size of output vector
recursion_limit= obsolete synonym for depth_limit
- replace= specify a replacement string
+ replace= specify a replacement string
startchar show startchar when relevant
startoffset= same as offset=
substitute_callout use substitution callouts
@@ -1324,6 +1355,7 @@ pattern, but can be overridden by modifiers on the subject.
substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY
substitute_skip= skip substitution number n
substitute_stop= skip substitution number n and greater
+ substitute_subject= specify a different subject for substitution
substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY
zero_terminate pass the subject as zero-terminated
@@ -1446,13 +1478,9 @@ difference to the matching process if the pattern begins with a lookbehind
assertion (including \eb or \eB).
.P
If an empty string is matched, the next match is done with the
-PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search for
-another, non-empty, match at the same point in the subject. If this match
-fails, the start offset is advanced, and the normal match is retried. This
-imitates the way Perl handles such cases when using the \fB/g\fP modifier or
-the \fBsplit()\fP function. Normally, the start offset is advanced by one
-character, but if the newline convention recognizes CRLF as a newline, and the
-current character is CR followed by LF, an advance of two characters occurs.
+PCRE2_NOTEMPTY_ATSTART flag set, in order to search for another, non-empty,
+match at the same point in the subject. This imitates the way Perl handles such
+cases when using the \fB/g\fP modifier or the \fBsplit()\fP function.
.
.
.SS "Testing substring extraction functions"
@@ -1540,7 +1568,7 @@ character. Here is an example that tests the edge case:
123abc123\e=replace=[10]XYZ
1: 123XYZ123
123abc123\e=replace=[9]XYZ
- Failed: error -47: no more memory
+ Failed: error -48: no more memory
.sp
The default action of \fBpcre2_substitute()\fP is to return
PCRE2_ERROR_NOMEMORY when the output buffer is too small. However, if the
@@ -1553,11 +1581,17 @@ for the trailing zero) as part of the error message. For example:
.sp
/abc/substitute_overflow_length
123abc123\e=replace=[9]XYZ
- Failed: error -47: no more memory: 10 code units are needed
+ Failed: error -48: no more memory: 10 code units are needed
.sp
A replacement string is ignored with POSIX and DFA matching. Specifying partial
matching provokes an error return ("bad option value") from
\fBpcre2_substitute()\fP.
+.sp
+The \fBsubstitute_subject\fP modifier may be used to test the use of the PCRE2
+API, in which a client calls \fBpcre2_match()\fP followed by \fBpcre2_substitute()\fP
+with PCRE2_SUBSTITUTE_MATCHED, but the client performs an unexpected and
+unsupported modification of the subject buffer in-place, in between the match
+and substitution.
.
.
.SS "Testing substitute callouts"
@@ -2241,6 +2275,6 @@ Cambridge, England.
.rs
.sp
.nf
-Last updated: 26 December 2024
+Last updated: 12 October 2025
Copyright (c) 1997-2024 University of Cambridge.
.fi
diff --git a/doc/pcre2test.txt b/doc/pcre2test.txt
index 5df8c71..b868b91 100644
--- a/doc/pcre2test.txt
+++ b/doc/pcre2test.txt
@@ -132,10 +132,6 @@ COMMAND LINE OPTIONS
as RunTest. The following options output the value and set
the exit code as indicated:
- ebcdic-nl the code for LF (= NL) in an EBCDIC environment:
- either 0x15 or 0x25
- 0 if used in an ASCII/Unicode environment
- exit code is always 0
linksize the configured internal link size (2, 3, or 4)
exit code is set to the link size
newline the default newline setting:
@@ -150,6 +146,12 @@ COMMAND LINE OPTIONS
backslash-C \C is supported (not locked out)
ebcdic compiled for an EBCDIC environment
+ ebcdic-io if PCRE2 is compiled for EBCDIC, whether
+ pcre2test's input and
+ output is EBCDIC or ASCII
+ ebcdic-nl25 if PCRE2 is compiled for EBCDIC, whether NL (=
+ LF) is 0x25
+ (otherwise it is 0x15, the default)
jit just-in-time support is available
pcre2-16 the 16-bit library was built
pcre2-32 the 32-bit library was built
@@ -157,9 +159,9 @@ COMMAND LINE OPTIONS
unicode Unicode support is available
Note that the availability of JIT support in the library does
- not guarantee that it can actually be used because in some
- environments it is unable to allocate executable memory. The
- option "jitusable" gives more detailed information. It re-
+ not guarantee that it can actually be used because in some
+ environments it is unable to allocate executable memory. The
+ option "jitusable" gives more detailed information. It re-
turns one of the following values:
0 JIT is available and usable
@@ -167,9 +169,14 @@ COMMAND LINE OPTIONS
2 JIT is not available
3 Unexpected return from test call to pcre2_jit_compile()
- If an unknown option is given, an error message is output;
+ If an unknown option is given, an error message is output;
the exit code is 0.
+ --colo[u]r[=]
+ By default, the output is coloured if the output file is a
+ terminal (auto). Force or suppress output of ANSI colour es-
+ capes with always and never respectively.
+
-d Behave as if each pattern has the debug modifier; the inter-
nal form and information about the compiled pattern is output
after compilation; -d is equivalent to -b -i.
@@ -178,48 +185,57 @@ COMMAND LINE OPTIONS
is done using the pcre2_dfa_match() function instead of the
default pcre2_match().
+ -E Run in "preprocess only" mode (similar to "gcc -E"). The "#if
+ ... #endif" commands are processed, and all other lines are
+ printed verbatim.
+
-error number[,number,...]
- Call pcre2_get_error_message() for each of the error numbers
- in the comma-separated list, display the resulting messages
- on the standard output, then exit with zero exit code. The
- numbers may be positive or negative. This is a convenience
+ Call pcre2_get_error_message() for each of the error numbers
+ in the comma-separated list, display the resulting messages
+ on the standard output, then exit with zero exit code. The
+ numbers may be positive or negative. This is a convenience
facility for PCRE2 maintainers.
-help Output a brief summary these options and then exit.
- -i Behave as if each pattern has the info modifier; information
+ -i Behave as if each pattern has the info modifier; information
about the compiled pattern is given after compilation.
- -jit Behave as if each pattern line has the jit modifier; after
- successful compilation, each pattern is passed to the just-
+ -jit Behave as if each pattern line has the jit modifier; after
+ successful compilation, each pattern is passed to the just-
in-time compiler, if available.
- -jitfast Behave as if each pattern line has the jitfast modifier; af-
- ter successful compilation, each pattern is passed to the
+ -jitfast Behave as if each pattern line has the jitfast modifier; af-
+ ter successful compilation, each pattern is passed to the
just-in-time compiler, if available, and each subject line is
passed directly to the JIT matcher via its "fast path".
-jitverify
- Behave as if each pattern line has the jitverify modifier;
- after successful compilation, each pattern is passed to the
- just-in-time compiler, if available, and the use of JIT for
+ Behave as if each pattern line has the jitverify modifier;
+ after successful compilation, each pattern is passed to the
+ just-in-time compiler, if available, and the use of JIT for
matching is verified.
-LM List modifiers: write a list of available pattern and subject
- modifiers to the standard output, then exit with zero exit
- code. All other options are ignored. If both -C and any -Lx
+ modifiers to the standard output, then exit with zero exit
+ code. All other options are ignored. If both -C and any -Lx
options are present, whichever is first is recognized.
- -LP List properties: write a list of recognized Unicode proper-
- ties to the standard output, then exit with zero exit code.
+ -LP List properties: write a list of recognized Unicode proper-
+ ties to the standard output, then exit with zero exit code.
All other options are ignored. If both -C and any -Lx options
are present, whichever is first is recognized.
-LS List scripts: write a list of recognized Unicode script names
- to the standard output, then exit with zero exit code. All
+ to the standard output, then exit with zero exit code. All
other options are ignored. If both -C and any -Lx options are
present, whichever is first is recognized.
+ -malloc Exercise malloc() failures, by first counting the number of
+ calls made to malloc during pattern compilation and matching,
+ then re-running the compilation and matching that many times,
+ exercising a failure of each malloc() call.
+
-pattern modifier-list
Behave as if each pattern line contains the given modifiers.
@@ -247,166 +263,181 @@ COMMAND LINE OPTIONS
a run, the total times for all compiles and matches are out-
put.
+ -unittest Run a fixed set of additional tests of the PCRE2 API which
+ are not driven by the test input files, and then exit.
+
-version Output the PCRE2 version number and then exit.
DESCRIPTION
- If pcre2test is given two filename arguments, it reads from the first
+ If pcre2test is given two filename arguments, it reads from the first
and writes to the second. If the first name is "-", input is taken from
- the standard input. If pcre2test is given only one argument, it reads
+ the standard input. If pcre2test is given only one argument, it reads
from that file and writes to stdout. Otherwise, it reads from stdin and
writes to stdout.
- When pcre2test is built, a configuration option can specify that it
- should be linked with the libreadline or libedit library. When this is
- done, if the input is from a terminal, it is read using the readline()
+ When pcre2test is built, a configuration option can specify that it
+ should be linked with the libreadline or libedit library. When this is
+ done, if the input is from a terminal, it is read using the readline()
function. This provides line-editing and history facilities. The output
from the -help option states whether or not readline() will be used.
- The program handles any number of tests, each of which consists of a
- set of input lines. Each set starts with a regular expression pattern,
+ The program handles any number of tests, each of which consists of a
+ set of input lines. Each set starts with a regular expression pattern,
followed by any number of subject lines to be matched against that pat-
tern. In between sets of test data, command lines that begin with # may
appear. This file format, with some restrictions, can also be processed
- by the perltest.sh script that is distributed with PCRE2 as a means of
+ by the perltest.sh script that is distributed with PCRE2 as a means of
checking that the behaviour of PCRE2 and Perl is the same. For a speci-
- fication of perltest.sh, see the comments near its beginning. See also
+ fication of perltest.sh, see the comments near its beginning. See also
the #perltest command below.
When the input is a terminal, pcre2test prompts for each line of input,
- using "re>" to prompt for regular expression patterns, and "data>" to
- prompt for subject lines. Command lines starting with # can be entered
+ using "re>" to prompt for regular expression patterns, and "data>" to
+ prompt for subject lines. Command lines starting with # can be entered
only in response to the "re>" prompt.
- Each subject line is matched separately and independently. If you want
+ Each subject line is matched separately and independently. If you want
to do multi-line matches, you have to use the \n escape sequence (or \r
- or \r\n, etc., depending on the newline setting) in a single line of
- input to encode the newline sequences. There is no limit on the length
- of subject lines; the input buffer is automatically extended if it is
- too small. There are replication features that makes it possible to
- generate long repetitive pattern or subject lines without having to
+ or \r\n, etc., depending on the newline setting) in a single line of
+ input to encode the newline sequences. There is no limit on the length
+ of subject lines; the input buffer is automatically extended if it is
+ too small. There are replication features that makes it possible to
+ generate long repetitive pattern or subject lines without having to
supply them explicitly.
- An empty line or the end of the file signals the end of the subject
- lines for a test, at which point a new pattern or command line is ex-
+ An empty line or the end of the file signals the end of the subject
+ lines for a test, at which point a new pattern or command line is ex-
pected if there is still input to be read.
COMMAND LINES
- In between sets of test data, a line that begins with # is interpreted
+ In between sets of test data, a line that begins with # is interpreted
as a command line. If the first character is followed by white space or
- an exclamation mark, the line is treated as a comment, and ignored.
+ an exclamation mark, the line is treated as a comment, and ignored.
Otherwise, the following commands are recognized:
#forbid_utf
- Subsequent patterns automatically have the PCRE2_NEVER_UTF and
- PCRE2_NEVER_UCP options set, which locks out the use of the PCRE2_UTF
- and PCRE2_UCP options and the use of (*UTF) and (*UCP) at the start of
- patterns. This command also forces an error if a subsequent pattern
- contains any occurrences of \P, \p, or \X, which are still supported
- when PCRE2_UTF is not set, but which require Unicode property support
+ Subsequent patterns automatically have the PCRE2_NEVER_UTF and
+ PCRE2_NEVER_UCP options set, which locks out the use of the PCRE2_UTF
+ and PCRE2_UCP options and the use of (*UTF) and (*UCP) at the start of
+ patterns. This command also forces an error if a subsequent pattern
+ contains any occurrences of \P, \p, or \X, which are still supported
+ when PCRE2_UTF is not set, but which require Unicode property support
to be included in the library.
- This is a trigger guard that is used in test files to ensure that UTF
- or Unicode property tests are not accidentally added to files that are
- used when Unicode support is not included in the library. Setting
- PCRE2_NEVER_UTF and PCRE2_NEVER_UCP as a default can also be obtained
- by the use of #pattern; the difference is that #forbid_utf cannot be
- unset, and the automatic options are not displayed in pattern informa-
+ This is a trigger guard that is used in test files to ensure that UTF
+ or Unicode property tests are not accidentally added to files that are
+ used when Unicode support is not included in the library. Setting
+ PCRE2_NEVER_UTF and PCRE2_NEVER_UCP as a default can also be obtained
+ by the use of #pattern; the difference is that #forbid_utf cannot be
+ unset, and the automatic options are not displayed in pattern informa-
tion, to avoid cluttering up test output.
#load
This command is used to load a set of precompiled patterns from a file,
- as described in the section entitled "Saving and restoring compiled
+ as described in the section entitled "Saving and restoring compiled
patterns" below.
#loadtables
- This command is used to load a set of binary character tables that can
- be accessed by the tables=3 qualifier. Such tables can be created by
+ This command is used to load a set of binary character tables that can
+ be accessed by the tables=3 qualifier. Such tables can be created by
the pcre2_dftables program with the -b option.
#newline_default []
- When PCRE2 is built, a default newline convention can be specified.
- This determines which characters and/or character pairs are recognized
+ When PCRE2 is built, a default newline convention can be specified.
+ This determines which characters and/or character pairs are recognized
as indicating a newline in a pattern or subject string. The default can
- be overridden when a pattern is compiled. The standard test files con-
- tain tests of various newline conventions, but the majority of the
- tests expect a single linefeed to be recognized as a newline by de-
- fault. Without special action the tests would fail when PCRE2 is com-
+ be overridden when a pattern is compiled. The standard test files con-
+ tain tests of various newline conventions, but the majority of the
+ tests expect a single linefeed to be recognized as a newline by de-
+ fault. Without special action the tests would fail when PCRE2 is com-
piled with either CR or CRLF as the default newline.
The #newline_default command specifies a list of newline types that are
- acceptable as the default. The types must be one of CR, LF, CRLF, ANY-
+ acceptable as the default. The types must be one of CR, LF, CRLF, ANY-
CRLF, ANY, or NUL (in upper or lower case), for example:
#newline_default LF Any anyCRLF
If the default newline is in the list, this command has no effect. Oth-
- erwise, except when testing the POSIX API, a newline modifier that
+ erwise, except when testing the POSIX API, a newline modifier that
specifies the first newline convention in the list (LF in the above ex-
- ample) is added to any pattern that does not already have a newline
+ ample) is added to any pattern that does not already have a newline
modifier. If the newline list is empty, the feature is turned off. This
command is present in a number of the standard test input files.
- When the POSIX API is being tested there is no way to override the de-
+ When the POSIX API is being tested there is no way to override the de-
fault newline convention, though it is possible to set the newline con-
- vention from within the pattern. A warning is given if the posix or
- posix_nosub modifier is used when #newline_default would set a default
+ vention from within the pattern. A warning is given if the posix or
+ posix_nosub modifier is used when #newline_default would set a default
for the non-POSIX API.
#pattern
- This command sets a default modifier list that applies to all subse-
+ This command sets a default modifier list that applies to all subse-
quent patterns. Modifiers on a pattern can change these settings.
#perltest
- This line is used in test files that can also be processed by perl-
- test.sh to confirm that Perl gives the same results as PCRE2. Subse-
- quent tests are checked for the use of pcre2test features that are in-
+ This line is used in test files that can also be processed by perl-
+ test.sh to confirm that Perl gives the same results as PCRE2. Subse-
+ quent tests are checked for the use of pcre2test features that are in-
compatible with the perltest.sh script.
- Patterns must use '/' as their delimiter, and only certain modifiers
- are supported. Comment lines, #pattern commands, and #subject commands
- that set or unset "mark" are recognized and acted on. The #perltest,
- #forbid_utf, and #newline_default commands, which are needed in the
+ Patterns must use '/' as their delimiter, and only certain modifiers
+ are supported. Comment lines, #pattern commands, and #subject commands
+ that set or unset "mark" are recognized and acted on. The #perltest,
+ #forbid_utf, and #newline_default commands, which are needed in the
relevant pcre2test files, are silently ignored. All other command lines
- are ignored, but give a warning message. The #perltest command helps
- detect tests that are accidentally put in the wrong file or use the
- wrong delimiter. For more details of the perltest.sh script see the
+ are ignored, but give a warning message. The #perltest command helps
+ detect tests that are accidentally put in the wrong file or use the
+ wrong delimiter. For more details of the perltest.sh script see the
comments it contains.
#pop []
#popcopy []
- These commands are used to manipulate the stack of compiled patterns,
- as described in the section entitled "Saving and restoring compiled
+ These commands are used to manipulate the stack of compiled patterns,
+ as described in the section entitled "Saving and restoring compiled
patterns" below.
#save
- This command is used to save a set of compiled patterns to a file, as
- described in the section entitled "Saving and restoring compiled pat-
+ This command is used to save a set of compiled patterns to a file, as
+ described in the section entitled "Saving and restoring compiled pat-
terns" below.
#subject
- This command sets a default modifier list that applies to all subse-
- quent subject lines. Modifiers on a subject line can change these set-
+ This command sets a default modifier list that applies to all subse-
+ quent subject lines. Modifiers on a subject line can change these set-
tings.
+ #if CONDITION
+ ...
+ #endif
+
+ If CONDITION is true, then the command is printed, and its contents are
+ processed as normal, including printing the commandlines to the output.
+ If CONDITION is false, then all lines between the "#if" and "#endif"
+ are skipped and not printed. The CONDITION can be any of the conditions
+ which are tested by the "-C" commandline option and which set
+ pcre2test's exit code to a boolean value. The CONDITION may also be
+ preceded by "!".
+
MODIFIER SYNTAX
Modifier lists are used with both pattern and subject lines. Items in a
list are separated by commas followed by optional white space. Trailing
- whitespace in a modifier list is ignored. Some modifiers may be given
+ white space in a modifier list is ignored. Some modifiers may be given
for both patterns and subject lines, whereas others are valid only for
one or the other. Each modifier has a long name, for example "an-
chored", and some of them must be followed by an equals sign and a
@@ -534,7 +565,7 @@ SUBJECT LINE SYNTAX
abc\=notbol,notempty
- If the subject string is empty and \= is followed by whitespace, the
+ If the subject string is empty and \= is followed by white space, the
line is treated as a comment line, and is not used for matching. For
example:
@@ -1045,7 +1076,8 @@ PATTERN MODIFIERS
heapframes_size show match data heapframes size
jitstack= set size of JIT stack
mark show mark values
- replace= specify a replacement string
+ null_substitute_match_data substitute with NULL match data
+ replace= specify a replacement string
startchar show starting character when relevant
substitute_callout use substitution callouts
substitute_case_callout use substitution case callouts
@@ -1205,11 +1237,12 @@ SUBJECT MODIFIERS
null_context match with a NULL context
null_replacement substitute with NULL replacement
null_subject match with NULL subject
+ null_substitute_match_data substitute with NULL match data
offset= set starting offset
offset_limit= set offset limit
ovector= set size of output vector
recursion_limit= obsolete synonym for depth_limit
- replace= specify a replacement string
+ replace= specify a replacement string
startchar show startchar when relevant
startoffset= same as offset=
substitute_callout use substitution callouts
@@ -1221,34 +1254,36 @@ SUBJECT MODIFIERS
substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY
substitute_skip= skip substitution number n
substitute_stop= skip substitution number n and greater
+ substitute_subject= specify a different subject for sub-
+ stitution
substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY
zero_terminate pass the subject as zero-terminated
The effects of these modifiers are described in the following sections.
- When matching via the POSIX wrapper API, the aftertext, allaftertext,
- and ovector subject modifiers work as described below. All other modi-
+ When matching via the POSIX wrapper API, the aftertext, allaftertext,
+ and ovector subject modifiers work as described below. All other modi-
fiers are either ignored, with a warning message, or cause an error.
Showing more text
- The aftertext modifier requests that as well as outputting the part of
+ The aftertext modifier requests that as well as outputting the part of
the subject string that matched the entire pattern, pcre2test should in
addition output the remainder of the subject string. This is useful for
tests where the subject contains multiple copies of the same substring.
- The allaftertext modifier requests the same action for captured sub-
+ The allaftertext modifier requests the same action for captured sub-
strings as well as the main matched substring. In each case the remain-
der is output on the following line with a plus character following the
capture number.
- The allusedtext modifier requests that all the text that was consulted
- during a successful pattern match by the interpreter should be shown,
- for both full and partial matches. This feature is not supported for
- JIT matching, and if requested with JIT it is ignored (with a warning
- message). Setting this modifier affects the output if there is a look-
- behind at the start of a match, or, for a complete match, a lookahead
+ The allusedtext modifier requests that all the text that was consulted
+ during a successful pattern match by the interpreter should be shown,
+ for both full and partial matches. This feature is not supported for
+ JIT matching, and if requested with JIT it is ignored (with a warning
+ message). Setting this modifier affects the output if there is a look-
+ behind at the start of a match, or, for a complete match, a lookahead
at the end, or if \K is used in the pattern. Characters that precede or
- follow the start and end of the actual match are indicated in the out-
+ follow the start and end of the actual match are indicated in the out-
put by '<' or '>' characters underneath them. Here is an example:
re> /(?<=pqr)abc(?=xyz)/
@@ -1259,16 +1294,16 @@ SUBJECT MODIFIERS
Partial match: pqrabcxy
<<<
- The first, complete match shows that the matched string is "abc", with
- the preceding and following strings "pqr" and "xyz" having been con-
- sulted during the match (when processing the assertions). The partial
+ The first, complete match shows that the matched string is "abc", with
+ the preceding and following strings "pqr" and "xyz" having been con-
+ sulted during the match (when processing the assertions). The partial
match can indicate only the preceding string.
- The startchar modifier requests that the starting character for the
- match be indicated, if it is different to the start of the matched
+ The startchar modifier requests that the starting character for the
+ match be indicated, if it is different to the start of the matched
string. The only time when this occurs is when \K has been processed as
part of the match. In this situation, the output for the matched string
- is displayed from the starting character instead of from the match
+ is displayed from the starting character instead of from the match
point, with circumflex characters under the earlier characters. For ex-
ample:
@@ -1277,7 +1312,7 @@ SUBJECT MODIFIERS
0: abcxyz
^^^
- Unlike allusedtext, the startchar modifier can be used with JIT. How-
+ Unlike allusedtext, the startchar modifier can be used with JIT. How-
ever, these two modifiers are mutually exclusive.
Showing the value of all capture groups
@@ -1285,58 +1320,54 @@ SUBJECT MODIFIERS
The allcaptures modifier requests that the values of all potential cap-
tured parentheses be output after a match. By default, only those up to
the highest one actually used in the match are output (corresponding to
- the return code from pcre2_match()). Groups that did not take part in
- the match are output as "". This modifier is not relevant for
- DFA matching (which does no capturing) and does not apply when replace
+ the return code from pcre2_match()). Groups that did not take part in
+ the match are output as "". This modifier is not relevant for
+ DFA matching (which does no capturing) and does not apply when replace
is specified; it is ignored, with a warning message, if present.
Showing the entire ovector, for all outcomes
The allvector modifier requests that the entire ovector be shown, what-
ever the outcome of the match. Compare allcaptures, which shows only up
- to the maximum number of capture groups for the pattern, and then only
- for a successful complete non-DFA match. This modifier, which acts af-
- ter any match result, and also for DFA matching, provides a means of
- checking that there are no unexpected modifications to ovector fields.
- Before each match attempt, the ovector is filled with a special value,
- and if this is found in both elements of a capturing pair, "" is output. After a successful match, this applies to all
- groups after the maximum capture group for the pattern. In other cases
- it applies to the entire ovector. After a partial match, the first two
- elements are the only ones that should be set. After a DFA match, the
- amount of ovector that is used depends on the number of matches that
+ to the maximum number of capture groups for the pattern, and then only
+ for a successful complete non-DFA match. This modifier, which acts af-
+ ter any match result, and also for DFA matching, provides a means of
+ checking that there are no unexpected modifications to ovector fields.
+ Before each match attempt, the ovector is filled with a special value,
+ and if this is found in both elements of a capturing pair, "" is output. After a successful match, this applies to all
+ groups after the maximum capture group for the pattern. In other cases
+ it applies to the entire ovector. After a partial match, the first two
+ elements are the only ones that should be set. After a DFA match, the
+ amount of ovector that is used depends on the number of matches that
were found.
Testing pattern callouts
- A callout function is supplied when pcre2test calls the library match-
- ing functions, unless callout_none is specified. Its behaviour can be
- controlled by various modifiers listed above whose names begin with
- callout_. Details are given in the section entitled "Callouts" below.
- Testing callouts from pcre2_substitute() is described separately in
+ A callout function is supplied when pcre2test calls the library match-
+ ing functions, unless callout_none is specified. Its behaviour can be
+ controlled by various modifiers listed above whose names begin with
+ callout_. Details are given in the section entitled "Callouts" below.
+ Testing callouts from pcre2_substitute() is described separately in
"Testing the substitution function" below.
Finding all matches in a string
Searching for all possible matches within a subject can be requested by
- the global or altglobal modifier. After finding a match, the matching
- function is called again to search the remainder of the subject. The
- difference between global and altglobal is that the former uses the
- start_offset argument to pcre2_match() or pcre2_dfa_match() to start
- searching at a new point within the entire string (which is what Perl
+ the global or altglobal modifier. After finding a match, the matching
+ function is called again to search the remainder of the subject. The
+ difference between global and altglobal is that the former uses the
+ start_offset argument to pcre2_match() or pcre2_dfa_match() to start
+ searching at a new point within the entire string (which is what Perl
does), whereas the latter passes over a shortened subject. This makes a
difference to the matching process if the pattern begins with a lookbe-
hind assertion (including \b or \B).
- If an empty string is matched, the next match is done with the
- PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search
- for another, non-empty, match at the same point in the subject. If this
- match fails, the start offset is advanced, and the normal match is re-
- tried. This imitates the way Perl handles such cases when using the /g
- modifier or the split() function. Normally, the start offset is ad-
- vanced by one character, but if the newline convention recognizes CRLF
- as a newline, and the current character is CR followed by LF, an ad-
- vance of two characters occurs.
+ If an empty string is matched, the next match is done with the
+ PCRE2_NOTEMPTY_ATSTART flag set, in order to search for another, non-
+ empty, match at the same point in the subject. This imitates the way
+ Perl handles such cases when using the /g modifier or the split() func-
+ tion.
Testing substring extraction functions
@@ -1418,7 +1449,7 @@ SUBJECT MODIFIERS
123abc123\=replace=[10]XYZ
1: 123XYZ123
123abc123\=replace=[9]XYZ
- Failed: error -47: no more memory
+ Failed: error -48: no more memory
The default action of pcre2_substitute() is to return PCRE2_ER-
ROR_NOMEMORY when the output buffer is too small. However, if the
@@ -1432,12 +1463,18 @@ SUBJECT MODIFIERS
/abc/substitute_overflow_length
123abc123\=replace=[9]XYZ
- Failed: error -47: no more memory: 10 code units are needed
+ Failed: error -48: no more memory: 10 code units are needed
A replacement string is ignored with POSIX and DFA matching. Specifying
partial matching provokes an error return ("bad option value") from
pcre2_substitute().
+ The substitute_subject modifier may be used to test the use of the
+ PCRE2 API, in which a client calls pcre2_match() followed by pcre2_sub-
+ stitute() with PCRE2_SUBSTITUTE_MATCHED, but the client performs an un-
+ expected and unsupported modification of the subject buffer in-place,
+ in between the match and substitution.
+
Testing substitute callouts
If the substitute_callout modifier is set, a substitution callout func-
@@ -2061,8 +2098,8 @@ AUTHOR
REVISION
- Last updated: 26 December 2024
+ Last updated: 12 October 2025
Copyright (c) 1997-2024 University of Cambridge.
-PCRE2 10.46 26 December 2024 PCRE2TEST(1)
+PCRE2 10.47 12 October 2025 PCRE2TEST(1)
diff --git a/doc/pcre2unicode.3 b/doc/pcre2unicode.3
index 417baf3..cb9f519 100644
--- a/doc/pcre2unicode.3
+++ b/doc/pcre2unicode.3
@@ -1,4 +1,4 @@
-.TH PCRE2UNICODE 3 "27 November 2024" "PCRE2 10.46"
+.TH PCRE2UNICODE 3 "27 November 2024" "PCRE2 10.47"
.SH NAME
PCRE2 - Perl-compatible regular expressions (revised API)
.SH "UNICODE AND UTF SUPPORT"
diff --git a/libpcre2-16.pc.in b/libpcre2-16.pc.in
index bacb466..fe47b58 100644
--- a/libpcre2-16.pc.in
+++ b/libpcre2-16.pc.in
@@ -8,6 +8,7 @@ includedir=@includedir@
Name: libpcre2-16
Description: PCRE2 - Perl compatible regular expressions C library (2nd API) with 16 bit character support
Version: @PACKAGE_VERSION@
+License: BSD-3-Clause WITH PCRE2-exception
Libs: -L${libdir} -lpcre2-16@LIB_POSTFIX@
Libs.private: @PTHREAD_CFLAGS@ @PTHREAD_LIBS@
Cflags: -I${includedir} @PCRE2_STATIC_CFLAG@
diff --git a/libpcre2-32.pc.in b/libpcre2-32.pc.in
index 06241f0..f2e9192 100644
--- a/libpcre2-32.pc.in
+++ b/libpcre2-32.pc.in
@@ -8,6 +8,7 @@ includedir=@includedir@
Name: libpcre2-32
Description: PCRE2 - Perl compatible regular expressions C library (2nd API) with 32 bit character support
Version: @PACKAGE_VERSION@
+License: BSD-3-Clause WITH PCRE2-exception
Libs: -L${libdir} -lpcre2-32@LIB_POSTFIX@
Libs.private: @PTHREAD_CFLAGS@ @PTHREAD_LIBS@
Cflags: -I${includedir} @PCRE2_STATIC_CFLAG@
diff --git a/libpcre2-8.pc.in b/libpcre2-8.pc.in
index 246bb9e..8697a95 100644
--- a/libpcre2-8.pc.in
+++ b/libpcre2-8.pc.in
@@ -8,6 +8,7 @@ includedir=@includedir@
Name: libpcre2-8
Description: PCRE2 - Perl compatible regular expressions C library (2nd API) with 8 bit character support
Version: @PACKAGE_VERSION@
+License: BSD-3-Clause WITH PCRE2-exception
Libs: -L${libdir} -lpcre2-8@LIB_POSTFIX@
Libs.private: @PTHREAD_CFLAGS@ @PTHREAD_LIBS@
Cflags: -I${includedir} @PCRE2_STATIC_CFLAG@
diff --git a/libpcre2-posix.pc.in b/libpcre2-posix.pc.in
index 2f1e8f1..faac2c5 100644
--- a/libpcre2-posix.pc.in
+++ b/libpcre2-posix.pc.in
@@ -8,6 +8,7 @@ includedir=@includedir@
Name: libpcre2-posix
Description: Posix compatible interface to libpcre2-8
Version: @PACKAGE_VERSION@
+License: BSD-3-Clause WITH PCRE2-exception
Libs: -L${libdir} -lpcre2-posix@LIB_POSTFIX@
Cflags: -I${includedir} @PCRE2POSIX_CFLAG@
Requires.private: libpcre2-8
diff --git a/m4/ax_check_vscript.m4 b/m4/ax_check_vscript.m4
new file mode 100644
index 0000000..ed834ef
--- /dev/null
+++ b/m4/ax_check_vscript.m4
@@ -0,0 +1,153 @@
+# ===========================================================================
+# https://www.gnu.org/software/autoconf-archive/ax_check_vscript.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_CHECK_VSCRIPT
+#
+# DESCRIPTION
+#
+# Check whether the linker supports version scripts. Version scripts are
+# used when building shared libraries to bind symbols to version nodes
+# (helping to detect incompatibilities) or to limit the visibility of
+# non-public symbols.
+#
+# Output:
+#
+# If version scripts are supported, VSCRIPT_LDFLAGS will contain the
+# appropriate flag to pass to the linker. On GNU systems this would
+# typically be "-Wl,--version-script", and on Solaris it would typically
+# be "-Wl,-M".
+#
+# Two Automake conditionals are also set:
+#
+# HAVE_VSCRIPT is true if the linker supports version scripts with
+# entries that use simple wildcards, like "local: *".
+#
+# HAVE_VSCRIPT_COMPLEX is true if the linker supports version scripts with
+# pattern matching wildcards, like "global: Java_*".
+#
+# On systems that do not support symbol versioning, such as Mac OS X, both
+# conditionals will be false. They will also be false if the user passes
+# "--disable-symvers" on the configure command line.
+#
+# Example:
+#
+# configure.ac:
+#
+# AX_CHECK_VSCRIPT
+#
+# Makefile.am:
+#
+# if HAVE_VSCRIPT
+# libfoo_la_LDFLAGS += $(VSCRIPT_LDFLAGS),@srcdir@/libfoo.map
+# endif
+#
+# if HAVE_VSCRIPT_COMPLEX
+# libbar_la_LDFLAGS += $(VSCRIPT_LDFLAGS),@srcdir@/libbar.map
+# endif
+#
+# LICENSE
+#
+# Copyright (c) 2014 Kevin Cernekee
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 2.99 PCRE2
+
+# _AX_CHECK_VSCRIPT(flag, global-sym, action-if-link-succeeds, [junk-file=no])
+AC_DEFUN([_AX_CHECK_VSCRIPT], [
+ AC_LANG_PUSH([C])
+ ax_check_vscript_save_flags="$LDFLAGS"
+ echo "V1 { global: $2; local: *; };" > conftest.map
+ AS_IF([test x$4 = xyes], [
+ echo "{" >> conftest.map
+ ])
+ LDFLAGS="$LDFLAGS -Wl,$1,conftest.map"
+ AC_LINK_IFELSE([AC_LANG_PROGRAM([[int show, hide;]], [])], [$3])
+ LDFLAGS="$ax_check_vscript_save_flags"
+ rm -f conftest.map
+ AC_LANG_POP([C])
+]) dnl _AX_CHECK_VSCRIPT
+
+AC_DEFUN([AX_CHECK_VSCRIPT], [
+
+ AC_ARG_ENABLE([symvers],
+ AS_HELP_STRING([--disable-symvers],
+ [disable library symbol versioning [default=auto]]),
+ [want_symvers=$enableval],
+ [want_symvers=yes]
+ )
+
+ AS_IF([test x$want_symvers = xyes], [
+
+ dnl First test --version-script and -M with a simple wildcard.
+
+ AC_CACHE_CHECK([linker version script flag], ax_cv_check_vscript_flag, [
+ ax_cv_check_vscript_flag=unsupported
+ _AX_CHECK_VSCRIPT([--version-script], [show], [
+ ax_cv_check_vscript_flag=--version-script
+ ])
+ AS_IF([test x$ax_cv_check_vscript_flag = xunsupported], [
+ # PCRE2: Support for FreeBSD. Rather annoyingly, AC_LINK_IFELSE will
+ # only test linking executables, and in turn, on FreeBSD the main
+ # entrypoint will fail to link if you use "local: *" to hide the
+ # visibility of various shared symbols injected from /usr/lib/crt1.o.
+ # It's not at all pretty to hardcode those symbol names here, but I
+ # can't think of an obvious way to improve on this.
+ _AX_CHECK_VSCRIPT([--version-script], [show;environ;__progname], [
+ ax_cv_check_vscript_flag=--version-script
+ ])
+ ])
+ AS_IF([test x$ax_cv_check_vscript_flag = xunsupported], [
+ _AX_CHECK_VSCRIPT([-M], [show], [ax_cv_check_vscript_flag=-M])
+ ])
+
+ dnl The linker may interpret -M (no argument) as "produce a load map."
+ dnl If "-M conftest.map" doesn't fail when conftest.map contains
+ dnl obvious syntax errors, assume this is the case.
+
+ AS_IF([test x$ax_cv_check_vscript_flag != xunsupported], [
+ _AX_CHECK_VSCRIPT([$ax_cv_check_vscript_flag], [show],
+ [ax_cv_check_vscript_flag=unsupported], [yes])
+ ])
+ ])
+
+ dnl If the simple wildcard worked, retest with a complex wildcard.
+
+ AS_IF([test x$ax_cv_check_vscript_flag != xunsupported], [
+ ax_check_vscript_flag=$ax_cv_check_vscript_flag
+ AC_CACHE_CHECK([if version scripts can use complex wildcards],
+ ax_cv_check_vscript_complex_wildcards, [
+ ax_cv_check_vscript_complex_wildcards=no
+ _AX_CHECK_VSCRIPT([$ax_cv_check_vscript_flag], [sh*], [
+ ax_cv_check_vscript_complex_wildcards=yes])
+ ])
+ ax_check_vscript_complex_wildcards="$ax_cv_check_vscript_complex_wildcards"
+ ], [
+ ax_check_vscript_flag=
+ ax_check_vscript_complex_wildcards=no
+ ])
+ ], [
+ AC_MSG_CHECKING([linker version script flag])
+ AC_MSG_RESULT([disabled])
+
+ ax_check_vscript_flag=
+ ax_check_vscript_complex_wildcards=no
+ ])
+
+ AS_IF([test x$ax_check_vscript_flag != x], [
+ VSCRIPT_LDFLAGS="-Wl,$ax_check_vscript_flag"
+ AC_SUBST([VSCRIPT_LDFLAGS])
+ ])
+
+ AM_CONDITIONAL([HAVE_VSCRIPT],
+ [test x$ax_check_vscript_flag != x])
+ AM_CONDITIONAL([HAVE_VSCRIPT_COMPLEX],
+ [test x$ax_check_vscript_complex_wildcards = xyes])
+
+]) dnl AX_CHECK_VSCRIPT
diff --git a/m4/ax_pthread.m4 b/m4/ax_pthread.m4
index d90de34..9f35d13 100644
--- a/m4/ax_pthread.m4
+++ b/m4/ax_pthread.m4
@@ -1,5 +1,5 @@
# ===========================================================================
-# http://www.gnu.org/software/autoconf-archive/ax_pthread.html
+# https://www.gnu.org/software/autoconf-archive/ax_pthread.html
# ===========================================================================
#
# SYNOPSIS
@@ -14,24 +14,28 @@
# flags that are needed. (The user can also force certain compiler
# flags/libs to be tested by setting these environment variables.)
#
-# Also sets PTHREAD_CC to any special C compiler that is needed for
-# multi-threaded programs (defaults to the value of CC otherwise). (This
-# is necessary on AIX to use the special cc_r compiler alias.)
+# Also sets PTHREAD_CC and PTHREAD_CXX to any special C compiler that is
+# needed for multi-threaded programs (defaults to the value of CC
+# respectively CXX otherwise). (This is necessary on e.g. AIX to use the
+# special cc_r/CC_r compiler alias.)
#
# NOTE: You are assumed to not only compile your program with these flags,
-# but also link it with them as well. e.g. you should link with
+# but also to link with them as well. For example, you might link with
# $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
+# $PTHREAD_CXX $CXXFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
#
-# If you are only building threads programs, you may wish to use these
+# If you are only building threaded programs, you may wish to use these
# variables in your default LIBS, CFLAGS, and CC:
#
# LIBS="$PTHREAD_LIBS $LIBS"
# CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
+# CXXFLAGS="$CXXFLAGS $PTHREAD_CFLAGS"
# CC="$PTHREAD_CC"
+# CXX="$PTHREAD_CXX"
#
# In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant
-# has a nonstandard name, defines PTHREAD_CREATE_JOINABLE to that name
-# (e.g. PTHREAD_CREATE_UNDETACHED on AIX).
+# has a nonstandard name, this macro defines PTHREAD_CREATE_JOINABLE to
+# that name (e.g. PTHREAD_CREATE_UNDETACHED on AIX).
#
# Also HAVE_PTHREAD_PRIO_INHERIT is defined if pthread is found and the
# PTHREAD_PRIO_INHERIT symbol is defined when compiling with
@@ -55,6 +59,7 @@
#
# Copyright (c) 2008 Steven G. Johnson
# Copyright (c) 2011 Daniel Richard G.
+# Copyright (c) 2019 Marc Stevens
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
@@ -67,7 +72,7 @@
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program. If not, see .
+# with this program. If not, see .
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
@@ -82,35 +87,41 @@
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
-#serial 18
+#serial 31
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
AC_REQUIRE([AC_CANONICAL_HOST])
+AC_REQUIRE([AC_PROG_CC])
+AC_REQUIRE([AC_PROG_SED])
AC_LANG_PUSH([C])
ax_pthread_ok=no
# We used to check for pthread.h first, but this fails if pthread.h
-# requires special compiler flags (e.g. on True64 or Sequent).
+# requires special compiler flags (e.g. on Tru64 or Sequent).
# It gets checked for in the link test anyway.
# First of all, check if the user has set any of the PTHREAD_LIBS,
# etcetera environment variables, and if threads linking works using
# them:
-if test x"$PTHREAD_LIBS$PTHREAD_CFLAGS" != x; then
- save_CFLAGS="$CFLAGS"
+if test "x$PTHREAD_CFLAGS$PTHREAD_LIBS" != "x"; then
+ ax_pthread_save_CC="$CC"
+ ax_pthread_save_CFLAGS="$CFLAGS"
+ ax_pthread_save_LIBS="$LIBS"
+ AS_IF([test "x$PTHREAD_CC" != "x"], [CC="$PTHREAD_CC"])
+ AS_IF([test "x$PTHREAD_CXX" != "x"], [CXX="$PTHREAD_CXX"])
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
- save_LIBS="$LIBS"
LIBS="$PTHREAD_LIBS $LIBS"
- AC_MSG_CHECKING([for pthread_join in LIBS=$PTHREAD_LIBS with CFLAGS=$PTHREAD_CFLAGS])
- AC_TRY_LINK_FUNC(pthread_join, ax_pthread_ok=yes)
- AC_MSG_RESULT($ax_pthread_ok)
- if test x"$ax_pthread_ok" = xno; then
+ AC_MSG_CHECKING([for pthread_join using $CC $PTHREAD_CFLAGS $PTHREAD_LIBS])
+ AC_LINK_IFELSE([AC_LANG_CALL([], [pthread_join])], [ax_pthread_ok=yes])
+ AC_MSG_RESULT([$ax_pthread_ok])
+ if test "x$ax_pthread_ok" = "xno"; then
PTHREAD_LIBS=""
PTHREAD_CFLAGS=""
fi
- LIBS="$save_LIBS"
- CFLAGS="$save_CFLAGS"
+ CC="$ax_pthread_save_CC"
+ CFLAGS="$ax_pthread_save_CFLAGS"
+ LIBS="$ax_pthread_save_LIBS"
fi
# We must check for the threads library under a number of different
@@ -118,12 +129,14 @@ fi
# (e.g. DEC) have both -lpthread and -lpthreads, where one of the
# libraries is broken (non-POSIX).
-# Create a list of thread flags to try. Items starting with a "-" are
-# C compiler flags, and other items are library names, except for "none"
-# which indicates that we try without any flags at all, and "pthread-config"
-# which is a program returning the flags for the Pth emulation library.
+# Create a list of thread flags to try. Items with a "," contain both
+# C compiler flags (before ",") and linker flags (after ","). Other items
+# starting with a "-" are C compiler flags, and remaining items are
+# library names, except for "none" which indicates that we try without
+# any flags at all, and "pthread-config" which is a program returning
+# the flags for the Pth emulation library.
-ax_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
+ax_pthread_flags="pthreads none -Kthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
# The ordering *is* (sometimes) important. Some notes on the
# individual items follow:
@@ -132,68 +145,163 @@ ax_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mt
# none: in case threads are in libc; should be tried before -Kthread and
# other compiler flags to prevent continual compiler warnings
# -Kthread: Sequent (threads in libc, but -Kthread needed for pthread.h)
-# -kthread: FreeBSD kernel threads (preferred to -pthread since SMP-able)
-# lthread: LinuxThreads port on FreeBSD (also preferred to -pthread)
-# -pthread: Linux/gcc (kernel threads), BSD/gcc (userland threads)
-# -pthreads: Solaris/gcc
-# -mthreads: Mingw32/gcc, Lynx/gcc
+# -pthread: Linux/gcc (kernel threads), BSD/gcc (userland threads), Tru64
+# (Note: HP C rejects this with "bad form for `-t' option")
+# -pthreads: Solaris/gcc (Note: HP C also rejects)
# -mt: Sun Workshop C (may only link SunOS threads [-lthread], but it
-# doesn't hurt to check since this sometimes defines pthreads too;
-# also defines -D_REENTRANT)
-# ... -mt is also the pthreads flag for HP/aCC
+# doesn't hurt to check since this sometimes defines pthreads and
+# -D_REENTRANT too), HP C (must be checked before -lpthread, which
+# is present but should not be used directly; and before -mthreads,
+# because the compiler interprets this as "-mt" + "-hreads")
+# -mthreads: Mingw32/gcc, Lynx/gcc
# pthread: Linux, etcetera
# --thread-safe: KAI C++
# pthread-config: use pthread-config program (for GNU Pth library)
-case ${host_os} in
+case $host_os in
+
+ freebsd*)
+
+ # -kthread: FreeBSD kernel threads (preferred to -pthread since SMP-able)
+ # lthread: LinuxThreads port on FreeBSD (also preferred to -pthread)
+
+ ax_pthread_flags="-kthread lthread $ax_pthread_flags"
+ ;;
+
+ hpux*)
+
+ # From the cc(1) man page: "[-mt] Sets various -D flags to enable
+ # multi-threading and also sets -lpthread."
+
+ ax_pthread_flags="-mt -pthread pthread $ax_pthread_flags"
+ ;;
+
+ openedition*)
+
+ # IBM z/OS requires a feature-test macro to be defined in order to
+ # enable POSIX threads at all, so give the user a hint if this is
+ # not set. (We don't define these ourselves, as they can affect
+ # other portions of the system API in unpredictable ways.)
+
+ AC_EGREP_CPP([AX_PTHREAD_ZOS_MISSING],
+ [
+# if !defined(_OPEN_THREADS) && !defined(_UNIX03_THREADS)
+ AX_PTHREAD_ZOS_MISSING
+# endif
+ ],
+ [AC_MSG_WARN([IBM z/OS requires -D_OPEN_THREADS or -D_UNIX03_THREADS to enable pthreads support.])])
+ ;;
+
solaris*)
# On Solaris (at least, for some versions), libc contains stubbed
# (non-functional) versions of the pthreads routines, so link-based
- # tests will erroneously succeed. (We need to link with -pthreads/-mt/
- # -lpthread.) (The stubs are missing pthread_cleanup_push, or rather
- # a function called by this macro, so we could check for that, but
- # who knows whether they'll stub that too in a future libc.) So,
- # we'll just look for -pthreads and -lpthread first:
+ # tests will erroneously succeed. (N.B.: The stubs are missing
+ # pthread_cleanup_push, or rather a function called by this macro,
+ # so we could check for that, but who knows whether they'll stub
+ # that too in a future libc.) So we'll check first for the
+ # standard Solaris way of linking pthreads (-mt -lpthread).
- ax_pthread_flags="-pthreads pthread -mt -pthread $ax_pthread_flags"
+ ax_pthread_flags="-mt,-lpthread pthread $ax_pthread_flags"
;;
+esac
- darwin*)
- ax_pthread_flags="-pthread $ax_pthread_flags"
+# Are we compiling with Clang?
+
+AC_CACHE_CHECK([whether $CC is Clang],
+ [ax_cv_PTHREAD_CLANG],
+ [ax_cv_PTHREAD_CLANG=no
+ # Note that Autoconf sets GCC=yes for Clang as well as GCC
+ if test "x$GCC" = "xyes"; then
+ AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG],
+ [/* Note: Clang 2.7 lacks __clang_[a-z]+__ */
+# if defined(__clang__) && defined(__llvm__)
+ AX_PTHREAD_CC_IS_CLANG
+# endif
+ ],
+ [ax_cv_PTHREAD_CLANG=yes])
+ fi
+ ])
+ax_pthread_clang="$ax_cv_PTHREAD_CLANG"
+
+
+# GCC generally uses -pthread, or -pthreads on some platforms (e.g. SPARC)
+
+# Note that for GCC and Clang -pthread generally implies -lpthread,
+# except when -nostdlib is passed.
+# This is problematic using libtool to build C++ shared libraries with pthread:
+# [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460
+# [2] https://bugzilla.redhat.com/show_bug.cgi?id=661333
+# [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=468555
+# To solve this, first try -pthread together with -lpthread for GCC
+
+AS_IF([test "x$GCC" = "xyes"],
+ [ax_pthread_flags="-pthread,-lpthread -pthread -pthreads $ax_pthread_flags"])
+
+# Clang takes -pthread (never supported any other flag), but we'll try with -lpthread first
+
+AS_IF([test "x$ax_pthread_clang" = "xyes"],
+ [ax_pthread_flags="-pthread,-lpthread -pthread"])
+
+
+# The presence of a feature test macro requesting re-entrant function
+# definitions is, on some systems, a strong hint that pthreads support is
+# correctly enabled
+
+case $host_os in
+ darwin* | hpux* | linux* | osf* | solaris*)
+ ax_pthread_check_macro="_REENTRANT"
+ ;;
+
+ aix*)
+ ax_pthread_check_macro="_THREAD_SAFE"
+ ;;
+
+ *)
+ ax_pthread_check_macro="--"
;;
esac
+AS_IF([test "x$ax_pthread_check_macro" = "x--"],
+ [ax_pthread_check_cond=0],
+ [ax_pthread_check_cond="!defined($ax_pthread_check_macro)"])
+
-if test x"$ax_pthread_ok" = xno; then
-for flag in $ax_pthread_flags; do
+if test "x$ax_pthread_ok" = "xno"; then
+for ax_pthread_try_flag in $ax_pthread_flags; do
- case $flag in
+ case $ax_pthread_try_flag in
none)
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
+ *,*)
+ PTHREAD_CFLAGS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\1/"`
+ PTHREAD_LIBS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\2/"`
+ AC_MSG_CHECKING([whether pthreads work with "$PTHREAD_CFLAGS" and "$PTHREAD_LIBS"])
+ ;;
+
-*)
- AC_MSG_CHECKING([whether pthreads work with $flag])
- PTHREAD_CFLAGS="$flag"
+ AC_MSG_CHECKING([whether pthreads work with $ax_pthread_try_flag])
+ PTHREAD_CFLAGS="$ax_pthread_try_flag"
;;
pthread-config)
- AC_CHECK_PROG(ax_pthread_config, pthread-config, yes, no)
- if test x"$ax_pthread_config" = xno; then continue; fi
+ AC_CHECK_PROG([ax_pthread_config], [pthread-config], [yes], [no])
+ AS_IF([test "x$ax_pthread_config" = "xno"], [continue])
PTHREAD_CFLAGS="`pthread-config --cflags`"
PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`"
;;
*)
- AC_MSG_CHECKING([for the pthreads library -l$flag])
- PTHREAD_LIBS="-l$flag"
+ AC_MSG_CHECKING([for the pthreads library -l$ax_pthread_try_flag])
+ PTHREAD_LIBS="-l$ax_pthread_try_flag"
;;
esac
- save_LIBS="$LIBS"
- save_CFLAGS="$CFLAGS"
- LIBS="$PTHREAD_LIBS $LIBS"
+ ax_pthread_save_CFLAGS="$CFLAGS"
+ ax_pthread_save_LIBS="$LIBS"
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
+ LIBS="$PTHREAD_LIBS $LIBS"
# Check for various functions. We must include pthread.h,
# since some functions may be macros. (On the Sequent, we
@@ -204,8 +312,18 @@ for flag in $ax_pthread_flags; do
# pthread_cleanup_push because it is one of the few pthread
# functions on Solaris that doesn't have a non-functional libc stub.
# We try pthread_create on general principles.
+
AC_LINK_IFELSE([AC_LANG_PROGRAM([#include
- static void routine(void *a) { a = 0; }
+# if $ax_pthread_check_cond
+# error "$ax_pthread_check_macro must be defined"
+# endif
+ static void *some_global = NULL;
+ static void routine(void *a)
+ {
+ /* To avoid any unused-parameter or
+ unused-but-set-parameter warning. */
+ some_global = a;
+ }
static void *start_routine(void *a) { return a; }],
[pthread_t th; pthread_attr_t attr;
pthread_create(&th, 0, start_routine, 0);
@@ -213,93 +331,188 @@ for flag in $ax_pthread_flags; do
pthread_attr_init(&attr);
pthread_cleanup_push(routine, 0);
pthread_cleanup_pop(0) /* ; */])],
- [ax_pthread_ok=yes],
- [])
+ [ax_pthread_ok=yes],
+ [])
- LIBS="$save_LIBS"
- CFLAGS="$save_CFLAGS"
+ CFLAGS="$ax_pthread_save_CFLAGS"
+ LIBS="$ax_pthread_save_LIBS"
- AC_MSG_RESULT($ax_pthread_ok)
- if test "x$ax_pthread_ok" = xyes; then
- break;
- fi
+ AC_MSG_RESULT([$ax_pthread_ok])
+ AS_IF([test "x$ax_pthread_ok" = "xyes"], [break])
PTHREAD_LIBS=""
PTHREAD_CFLAGS=""
done
fi
+
+# Clang needs special handling, because older versions handle the -pthread
+# option in a rather... idiosyncratic way
+
+if test "x$ax_pthread_clang" = "xyes"; then
+
+ # Clang takes -pthread; it has never supported any other flag
+
+ # (Note 1: This will need to be revisited if a system that Clang
+ # supports has POSIX threads in a separate library. This tends not
+ # to be the way of modern systems, but it's conceivable.)
+
+ # (Note 2: On some systems, notably Darwin, -pthread is not needed
+ # to get POSIX threads support; the API is always present and
+ # active. We could reasonably leave PTHREAD_CFLAGS empty. But
+ # -pthread does define _REENTRANT, and while the Darwin headers
+ # ignore this macro, third-party headers might not.)
+
+ # However, older versions of Clang make a point of warning the user
+ # that, in an invocation where only linking and no compilation is
+ # taking place, the -pthread option has no effect ("argument unused
+ # during compilation"). They expect -pthread to be passed in only
+ # when source code is being compiled.
+ #
+ # Problem is, this is at odds with the way Automake and most other
+ # C build frameworks function, which is that the same flags used in
+ # compilation (CFLAGS) are also used in linking. Many systems
+ # supported by AX_PTHREAD require exactly this for POSIX threads
+ # support, and in fact it is often not straightforward to specify a
+ # flag that is used only in the compilation phase and not in
+ # linking. Such a scenario is extremely rare in practice.
+ #
+ # Even though use of the -pthread flag in linking would only print
+ # a warning, this can be a nuisance for well-run software projects
+ # that build with -Werror. So if the active version of Clang has
+ # this misfeature, we search for an option to squash it.
+
+ AC_CACHE_CHECK([whether Clang needs flag to prevent "argument unused" warning when linking with -pthread],
+ [ax_cv_PTHREAD_CLANG_NO_WARN_FLAG],
+ [ax_cv_PTHREAD_CLANG_NO_WARN_FLAG=unknown
+ # Create an alternate version of $ac_link that compiles and
+ # links in two steps (.c -> .o, .o -> exe) instead of one
+ # (.c -> exe), because the warning occurs only in the second
+ # step
+ ax_pthread_save_ac_link="$ac_link"
+ ax_pthread_sed='s/conftest\.\$ac_ext/conftest.$ac_objext/g'
+ ax_pthread_link_step=`AS_ECHO(["$ac_link"]) | sed "$ax_pthread_sed"`
+ ax_pthread_2step_ac_link="($ac_compile) && (echo ==== >&5) && ($ax_pthread_link_step)"
+ ax_pthread_save_CFLAGS="$CFLAGS"
+ for ax_pthread_try in '' -Qunused-arguments -Wno-unused-command-line-argument unknown; do
+ AS_IF([test "x$ax_pthread_try" = "xunknown"], [break])
+ CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"
+ ac_link="$ax_pthread_save_ac_link"
+ AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
+ [ac_link="$ax_pthread_2step_ac_link"
+ AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
+ [break])
+ ])
+ done
+ ac_link="$ax_pthread_save_ac_link"
+ CFLAGS="$ax_pthread_save_CFLAGS"
+ AS_IF([test "x$ax_pthread_try" = "x"], [ax_pthread_try=no])
+ ax_cv_PTHREAD_CLANG_NO_WARN_FLAG="$ax_pthread_try"
+ ])
+
+ case "$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG" in
+ no | unknown) ;;
+ *) PTHREAD_CFLAGS="$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG $PTHREAD_CFLAGS" ;;
+ esac
+
+fi # $ax_pthread_clang = yes
+
+
+
# Various other checks:
-if test "x$ax_pthread_ok" = xyes; then
- save_LIBS="$LIBS"
- LIBS="$PTHREAD_LIBS $LIBS"
- save_CFLAGS="$CFLAGS"
+if test "x$ax_pthread_ok" = "xyes"; then
+ ax_pthread_save_CFLAGS="$CFLAGS"
+ ax_pthread_save_LIBS="$LIBS"
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
+ LIBS="$PTHREAD_LIBS $LIBS"
# Detect AIX lossage: JOINABLE attribute is called UNDETACHED.
- AC_MSG_CHECKING([for joinable pthread attribute])
- attr_name=unknown
- for attr in PTHREAD_CREATE_JOINABLE PTHREAD_CREATE_UNDETACHED; do
- AC_LINK_IFELSE([AC_LANG_PROGRAM([#include ],
- [int attr = $attr; return attr /* ; */])],
- [attr_name=$attr; break],
- [])
- done
- AC_MSG_RESULT($attr_name)
- if test "$attr_name" != PTHREAD_CREATE_JOINABLE; then
- AC_DEFINE_UNQUOTED(PTHREAD_CREATE_JOINABLE, $attr_name,
- [Define to necessary symbol if this constant
- uses a non-standard name on your system.])
- fi
+ AC_CACHE_CHECK([for joinable pthread attribute],
+ [ax_cv_PTHREAD_JOINABLE_ATTR],
+ [ax_cv_PTHREAD_JOINABLE_ATTR=unknown
+ for ax_pthread_attr in PTHREAD_CREATE_JOINABLE PTHREAD_CREATE_UNDETACHED; do
+ AC_LINK_IFELSE([AC_LANG_PROGRAM([#include ],
+ [int attr = $ax_pthread_attr; return attr /* ; */])],
+ [ax_cv_PTHREAD_JOINABLE_ATTR=$ax_pthread_attr; break],
+ [])
+ done
+ ])
+ AS_IF([test "x$ax_cv_PTHREAD_JOINABLE_ATTR" != "xunknown" && \
+ test "x$ax_cv_PTHREAD_JOINABLE_ATTR" != "xPTHREAD_CREATE_JOINABLE" && \
+ test "x$ax_pthread_joinable_attr_defined" != "xyes"],
+ [AC_DEFINE_UNQUOTED([PTHREAD_CREATE_JOINABLE],
+ [$ax_cv_PTHREAD_JOINABLE_ATTR],
+ [Define to necessary symbol if this constant
+ uses a non-standard name on your system.])
+ ax_pthread_joinable_attr_defined=yes
+ ])
- AC_MSG_CHECKING([if more special flags are required for pthreads])
- flag=no
- case ${host_os} in
- aix* | freebsd* | darwin*) flag="-D_THREAD_SAFE";;
- osf* | hpux*) flag="-D_REENTRANT";;
- solaris*)
- if test "$GCC" = "yes"; then
- flag="-D_REENTRANT"
- else
- flag="-mt -D_REENTRANT"
- fi
- ;;
- esac
- AC_MSG_RESULT(${flag})
- if test "x$flag" != xno; then
- PTHREAD_CFLAGS="$flag $PTHREAD_CFLAGS"
- fi
+ AC_CACHE_CHECK([whether more special flags are required for pthreads],
+ [ax_cv_PTHREAD_SPECIAL_FLAGS],
+ [ax_cv_PTHREAD_SPECIAL_FLAGS=no
+ case $host_os in
+ solaris*)
+ ax_cv_PTHREAD_SPECIAL_FLAGS="-D_POSIX_PTHREAD_SEMANTICS"
+ ;;
+ esac
+ ])
+ AS_IF([test "x$ax_cv_PTHREAD_SPECIAL_FLAGS" != "xno" && \
+ test "x$ax_pthread_special_flags_added" != "xyes"],
+ [PTHREAD_CFLAGS="$ax_cv_PTHREAD_SPECIAL_FLAGS $PTHREAD_CFLAGS"
+ ax_pthread_special_flags_added=yes])
AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT],
- ax_cv_PTHREAD_PRIO_INHERIT, [
- AC_LINK_IFELSE([
- AC_LANG_PROGRAM([[#include ]], [[int i = PTHREAD_PRIO_INHERIT;]])],
- [ax_cv_PTHREAD_PRIO_INHERIT=yes],
- [ax_cv_PTHREAD_PRIO_INHERIT=no])
+ [ax_cv_PTHREAD_PRIO_INHERIT],
+ [AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]],
+ [[int i = PTHREAD_PRIO_INHERIT;
+ return i;]])],
+ [ax_cv_PTHREAD_PRIO_INHERIT=yes],
+ [ax_cv_PTHREAD_PRIO_INHERIT=no])
])
- AS_IF([test "x$ax_cv_PTHREAD_PRIO_INHERIT" = "xyes"],
- AC_DEFINE([HAVE_PTHREAD_PRIO_INHERIT], 1, [Have PTHREAD_PRIO_INHERIT.]))
+ AS_IF([test "x$ax_cv_PTHREAD_PRIO_INHERIT" = "xyes" && \
+ test "x$ax_pthread_prio_inherit_defined" != "xyes"],
+ [AC_DEFINE([HAVE_PTHREAD_PRIO_INHERIT], [1], [Have PTHREAD_PRIO_INHERIT.])
+ ax_pthread_prio_inherit_defined=yes
+ ])
- LIBS="$save_LIBS"
- CFLAGS="$save_CFLAGS"
+ CFLAGS="$ax_pthread_save_CFLAGS"
+ LIBS="$ax_pthread_save_LIBS"
- # More AIX lossage: must compile with xlc_r or cc_r
- if test x"$GCC" != xyes; then
- AC_CHECK_PROGS(PTHREAD_CC, xlc_r cc_r, ${CC})
- else
- PTHREAD_CC=$CC
+ # More AIX lossage: compile with *_r variant
+ if test "x$GCC" != "xyes"; then
+ case $host_os in
+ aix*)
+ AS_CASE(["x/$CC"],
+ [x*/c89|x*/c89_128|x*/c99|x*/c99_128|x*/cc|x*/cc128|x*/xlc|x*/xlc_v6|x*/xlc128|x*/xlc128_v6],
+ [#handle absolute path differently from PATH based program lookup
+ AS_CASE(["x$CC"],
+ [x/*],
+ [
+ AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])
+ AS_IF([test "x${CXX}" != "x"], [AS_IF([AS_EXECUTABLE_P([${CXX}_r])],[PTHREAD_CXX="${CXX}_r"])])
+ ],
+ [
+ AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])
+ AS_IF([test "x${CXX}" != "x"], [AC_CHECK_PROGS([PTHREAD_CXX],[${CXX}_r],[$CXX])])
+ ]
+ )
+ ])
+ ;;
+ esac
fi
-else
- PTHREAD_CC="$CC"
fi
-AC_SUBST(PTHREAD_LIBS)
-AC_SUBST(PTHREAD_CFLAGS)
-AC_SUBST(PTHREAD_CC)
+test -n "$PTHREAD_CC" || PTHREAD_CC="$CC"
+test -n "$PTHREAD_CXX" || PTHREAD_CXX="$CXX"
+
+AC_SUBST([PTHREAD_LIBS])
+AC_SUBST([PTHREAD_CFLAGS])
+AC_SUBST([PTHREAD_CC])
+AC_SUBST([PTHREAD_CXX])
# Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND:
-if test x"$ax_pthread_ok" = xyes; then
- ifelse([$1],,AC_DEFINE(HAVE_PTHREAD,1,[Define if you have POSIX threads libraries and header files.]),[$1])
+if test "x$ax_pthread_ok" = "xyes"; then
+ ifelse([$1],,[AC_DEFINE([HAVE_PTHREAD],[1],[Define if you have POSIX threads libraries and header files.])],[$1])
:
else
ax_pthread_ok=no
diff --git a/m4/pcre2_visibility.m4 b/m4/pcre2_visibility.m4
index 03f4fba..1e4e819 100644
--- a/m4/pcre2_visibility.m4
+++ b/m4/pcre2_visibility.m4
@@ -34,10 +34,10 @@ AC_DEFUN([PCRE2_VISIBILITY],
AC_COMPILE_IFELSE(
[AC_LANG_PROGRAM([[]], [[]])],
[
- AC_COMPILE_IFELSE(
- [AC_LANG_PROGRAM([[]], [[ #warning e ]])],
+ AC_COMPILE_IFELSE(
+ [AC_LANG_PROGRAM([[]], [[ #warning e ]])],
[], [pcre2_cv_cc_vis_werror=yes]
- )
+ )
], [])
CFLAGS="$pcre2_save_CFLAGS"])
AC_MSG_RESULT([$pcre2_cv_cc_vis_werror])
@@ -74,9 +74,9 @@ AC_DEFUN([PCRE2_VISIBILITY],
then
VISIBILITY_CFLAGS="-fvisibility=hidden"
HAVE_VISIBILITY=1
- AC_DEFINE(PCRE2_EXPORT, [__attribute__ ((visibility ("default")))], [to make a symbol visible])
+ AC_DEFINE(PCRE2_EXPORT, [__attribute__ ((visibility ("default")))], [Define to the annotation for making a symbol visible.])
else
- AC_DEFINE(PCRE2_EXPORT, [], [to make a symbol visible])
+ AC_DEFINE(PCRE2_EXPORT, [], [Define to the annotation for making a symbol visible.])
fi
AC_SUBST([VISIBILITY_CFLAGS])
AC_SUBST([HAVE_VISIBILITY])
diff --git a/m4/pcre2_zos.m4 b/m4/pcre2_zos.m4
new file mode 100644
index 0000000..2e5dbd4
--- /dev/null
+++ b/m4/pcre2_zos.m4
@@ -0,0 +1,50 @@
+dnl Tests whether the compiler requires an additional flag in order to fail on
+dnl undefined headers.
+
+dnl The concept of setting this commandline flag was learned from patches and
+dnl mailing list discussions of the gnulib and gawk projects (credit to
+dnl Bruno Haible).
+
+AC_DEFUN([PCRE2_ZOS_FIXES],
+[
+ AC_CACHE_CHECK([for OS/390 (z/OS)], [pcre2_cv_os390],
+ [if test "`uname`" = "OS/390"; then
+ pcre2_cv_os390=yes
+ else
+ pcre2_cv_os390=no
+ fi])
+ if test "$pcre2_cv_os390" = "yes"; then
+ AC_CACHE_CHECK([whether the compiler supports -qhaltonmsg=CCN3296], [pcre2_cv_xlc_qhaltonmsg_support],
+ [save_CFLAGS="$CFLAGS"
+ CFLAGS="$CFLAGS -qhaltonmsg=CCN3296"
+ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([])],
+ [pcre2_cv_xlc_qhaltonmsg_support=yes],
+ [pcre2_cv_xlc_qhaltonmsg_support=no])
+ CFLAGS="$save_CFLAGS"
+ ])
+
+ AC_CACHE_CHECK([whether non-existent headers fail the compile], [pcre2_cv_xlc_nonexistent_fatal],
+ [AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]])],
+ [pcre2_cv_xlc_nonexistent_fatal=no],
+ [pcre2_cv_xlc_nonexistent_fatal=yes])
+ ])
+
+ if test "$pcre2_cv_xlc_nonexistent_fatal" = "no" && test "$pcre2_cv_xlc_qhaltonmsg_support" = "yes"; then
+ AC_CACHE_CHECK([whether -qhaltonmsg=CCN3296 fixes the non-existent-header issue], [pcre2_cv_xlc_qhaltonmsg_fixes],
+ [save_CFLAGS="$CFLAGS"
+ CFLAGS="$CFLAGS -qhaltonmsg=CCN3296"
+ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]])],
+ [pcre2_cv_xlc_qhaltonmsg_fixes=no],
+ [pcre2_cv_xlc_qhaltonmsg_fixes=yes])
+ CFLAGS="$save_CFLAGS"
+ ])
+
+ if test "$pcre2_cv_xlc_qhaltonmsg_fixes" = "no"; then
+ AC_MSG_ERROR([-qhaltonmsg=CCN3296 not effective on non-existent headers])
+ fi
+
+ CFLAGS="$CFLAGS -qhaltonmsg=CCN3296"
+ fi
+
+ fi
+])
diff --git a/perltest.sh b/perltest.sh
index 37d4511..65031ce 100755
--- a/perltest.sh
+++ b/perltest.sh
@@ -118,9 +118,10 @@ fi
# already have. NOTE: this is different to pcre2test where #pattern sets
# defaults which can be overridden on individual patterns. The #subject command
# may be used to set or unset a default "mark" modifier for data lines. This is
-# the only use of #subject that is supported. The #perltest, #forbid_utf, and
-# #newline_default commands, which are needed in the relevant pcre2test files,
-# are ignored. Any other #-command is ignored, with a warning message.
+# the only use of #subject that is supported. The #perltest, #forbid_utf,
+# #newline_default, and #if...#endif commands, which are needed in the relevant
+# pcre2test files, are ignored. Any other #-command is ignored, with a warning
+# message.
#
# The pattern lines should use only / as the delimiter. The other characters
# that pcre2test supports cause problems with this script.
@@ -247,7 +248,7 @@ for (;;)
}
elsif ($_ =~ /^#/)
{
- if ($_ !~ /^#newline_default|^#perltest|^#forbid_utf/)
+ if ($_ !~ /^#newline_default|^#perltest|^#forbid_utf|^#if|^#endif/)
{
printf $outfile "** Warning: #-command ignored: %s", $_;
}
diff --git a/src/config-cmake.h.in b/src/config-cmake.h.in
new file mode 100644
index 0000000..bcc62c0
--- /dev/null
+++ b/src/config-cmake.h.in
@@ -0,0 +1,55 @@
+/* config.h for CMake builds */
+
+#cmakedefine HAVE_ASSERT_H 1
+#cmakedefine HAVE_BUILTIN_ASSUME 1
+#cmakedefine HAVE_BUILTIN_MUL_OVERFLOW 1
+#cmakedefine HAVE_BUILTIN_UNREACHABLE 1
+#cmakedefine HAVE_ATTRIBUTE_UNINITIALIZED 1
+#cmakedefine HAVE_DIRENT_H 1
+#cmakedefine HAVE_SYS_STAT_H 1
+#cmakedefine HAVE_SYS_TYPES_H 1
+#cmakedefine HAVE_UNISTD_H 1
+#cmakedefine HAVE_WINDOWS_H 1
+
+#cmakedefine HAVE_MEMFD_CREATE 1
+#cmakedefine HAVE_SECURE_GETENV 1
+
+#cmakedefine SUPPORT_PCRE2_8 1
+#cmakedefine SUPPORT_PCRE2_16 1
+#cmakedefine SUPPORT_PCRE2_32 1
+#cmakedefine DISABLE_PERCENT_ZT 1
+
+#cmakedefine SUPPORT_LIBBZ2 1
+#cmakedefine SUPPORT_LIBEDIT 1
+#cmakedefine SUPPORT_LIBREADLINE 1
+#cmakedefine SUPPORT_LIBZ 1
+
+#cmakedefine SUPPORT_JIT 1
+#cmakedefine SLJIT_PROT_EXECUTABLE_ALLOCATOR 1
+#cmakedefine SUPPORT_PCRE2GREP_JIT 1
+#cmakedefine SUPPORT_PCRE2GREP_CALLOUT 1
+#cmakedefine SUPPORT_PCRE2GREP_CALLOUT_FORK 1
+#cmakedefine SUPPORT_UNICODE 1
+#cmakedefine SUPPORT_VALGRIND 1
+
+#cmakedefine BSR_ANYCRLF 1
+#cmakedefine EBCDIC 1
+#cmakedefine EBCDIC_NL25 1
+#cmakedefine EBCDIC_IGNORING_COMPILER 1
+#cmakedefine NEVER_BACKSLASH_C 1
+
+#define PCRE2_EXPORT @PCRE2_EXPORT@
+#define LINK_SIZE @PCRE2_LINK_SIZE@
+#define HEAP_LIMIT @PCRE2_HEAP_LIMIT@
+#define MATCH_LIMIT @PCRE2_MATCH_LIMIT@
+#define MATCH_LIMIT_DEPTH @PCRE2_MATCH_LIMIT_DEPTH@
+#define MAX_VARLOOKBEHIND @PCRE2_MAX_VARLOOKBEHIND@
+#define NEWLINE_DEFAULT @NEWLINE_DEFAULT@
+#define PARENS_NEST_LIMIT @PCRE2_PARENS_NEST_LIMIT@
+#define PCRE2GREP_BUFSIZE @PCRE2GREP_BUFSIZE@
+#define PCRE2GREP_MAX_BUFSIZE @PCRE2GREP_MAX_BUFSIZE@
+
+#define MAX_NAME_SIZE 128
+#define MAX_NAME_COUNT 10000
+
+/* end config.h for CMake builds */
diff --git a/src/config.h.generic b/src/config.h.generic
index e982933..48da31e 100644
--- a/src/config.h.generic
+++ b/src/config.h.generic
@@ -22,11 +22,7 @@ Boolean macros such as HAVE_STDLIB_H and SUPPORT_PCRE2_8 should either be
defined (conventionally to 1) for TRUE, and not defined at all for FALSE. All
such macros are listed as a commented #undef in config.h.generic. Macros such
as MATCH_LIMIT, whose actual value is relevant, have defaults defined, but are
-surrounded by #ifndef/#endif lines so that the value can be overridden by -D.
-
-PCRE2 uses memmove() if HAVE_MEMMOVE is defined; otherwise it uses bcopy() if
-HAVE_BCOPY is defined. If your system has neither bcopy() nor memmove(), make
-sure both macros are undefined; an emulation function will then be used. */
+surrounded by #ifndef/#endif lines so that the value can be overridden by -D. */
/* By default, the \R escape sequence matches any Unicode line ending
character or sequence of characters. If BSR_ANYCRLF is defined (to any
@@ -44,9 +40,14 @@ sure both macros are undefined; an emulation function will then be used. */
assumes that all input strings are in EBCDIC. If you do not define this
macro, PCRE2 will assume input strings are ASCII or UTF-8/16/32 Unicode. It
is not possible to build a version of PCRE2 that supports both EBCDIC and
- UTF-8/16/32. */
+ ASCII or UTF-8/16/32. */
/* #undef EBCDIC */
+/* To force an EBCDIC environment, define this macro to make the core PCRE2
+ library functions use EBCDIC codepage 1047, regardless of whether the
+ compiler supports it using C character literals. */
+/* #undef EBCDIC_IGNORING_COMPILER */
+
/* In an EBCDIC environment, define this macro to any value to arrange for the
NL character to be 0x25 instead of the default 0x15. NL plays the role that
LF does in an ASCII/Unicode environment. */
@@ -58,9 +59,6 @@ sure both macros are undefined; an emulation function will then be used. */
/* Define this if your compiler supports __attribute__((uninitialized)) */
/* #undef HAVE_ATTRIBUTE_UNINITIALIZED */
-/* Define to 1 if you have the `bcopy' function. */
-/* #undef HAVE_BCOPY */
-
/* Define this if your compiler provides __assume() */
/* #undef HAVE_BUILTIN_ASSUME */
@@ -94,9 +92,6 @@ sure both macros are undefined; an emulation function will then be used. */
/* Define to 1 if you have the `memfd_create' function. */
/* #undef HAVE_MEMFD_CREATE */
-/* Define to 1 if you have the `memmove' function. */
-/* #undef HAVE_MEMMOVE */
-
/* Define to 1 if you have the header file. */
/* #undef HAVE_MINIX_CONFIG_H */
@@ -133,9 +128,6 @@ sure both macros are undefined; an emulation function will then be used. */
/* Define to 1 if you have the header file. */
/* #undef HAVE_STDLIB_H */
-/* Define to 1 if you have the `strerror' function. */
-/* #undef HAVE_STRERROR */
-
/* Define to 1 if you have the header file. */
/* #undef HAVE_STRINGS_H */
@@ -255,7 +247,7 @@ sure both macros are undefined; an emulation function will then be used. */
#define PACKAGE_NAME "PCRE2"
/* Define to the full name and version of this package. */
-#define PACKAGE_STRING "PCRE2 10.46"
+#define PACKAGE_STRING "PCRE2 10.47"
/* Define to the one symbol short name of this package. */
#define PACKAGE_TARNAME "pcre2"
@@ -264,7 +256,7 @@ sure both macros are undefined; an emulation function will then be used. */
#define PACKAGE_URL ""
/* Define to the version of this package. */
-#define PACKAGE_VERSION "10.46"
+#define PACKAGE_VERSION "10.47"
/* The value of PARENS_NEST_LIMIT specifies the maximum depth of nested
parentheses (of any kind) in a pattern. This limits the amount of system
@@ -291,10 +283,24 @@ sure both macros are undefined; an emulation function will then be used. */
#define PCRE2GREP_MAX_BUFSIZE 1048576
#endif
+/* See PCRE2_EXP_DEFN; but this is applied to functions in the libpcre2-posix
+ library. */
+/* #undef PCRE2POSIX_EXP_DEFN */
+
+/* Define to any value if linking libpcre2-posix dynamically. Ideally, if both
+ static and shared libraries are being built, then PCRE2POSIX_SHARED would
+ be defined only for the shared build. Indeed, this is a requirement on
+ Windows. However, when building with Autoconf and libtool, we compile the
+ sources once only to create both the static and shared library, so in this
+ case, PCRE2POSIX_SHARED should only be defined if the shared library is
+ being built, regardless of whether or not the static library is also being
+ built. */
+/* #undef PCRE2POSIX_SHARED */
+
/* Define to any value to include debugging code. */
/* #undef PCRE2_DEBUG */
-/* to make a symbol visible */
+/* Define to the annotation for making a symbol visible. */
#define PCRE2_EXPORT
/* If you are compiling for a system other than a Unix-like system or
@@ -309,7 +315,12 @@ sure both macros are undefined; an emulation function will then be used. */
in the C sense, but which are internal to the library. */
/* #undef PCRE2_EXP_DEFN */
-/* Define to any value if linking statically (TODO: make nice with Libtool) */
+/* Define to any value if linking statically. Ideally, if both static and
+ shared libraries are being built, then PCRE2_STATIC would be defined only
+ for the static build. Indeed, this is a requirement on Windows. With
+ Autoconf and libtool however, it is idiomatic to compile the sources once
+ to create both the static and shared library, so in this case, PCRE2_STATIC
+ should only be defined if no shared library is being built. */
/* #undef PCRE2_STATIC */
/* Define to necessary symbol if this constant uses a non-standard name on
@@ -464,7 +475,7 @@ sure both macros are undefined; an emulation function will then be used. */
#endif
/* Version number of package */
-#define VERSION "10.46"
+#define VERSION "10.47"
/* Number of bits in a file offset, on hosts where this is settable. */
/* #undef _FILE_OFFSET_BITS */
diff --git a/src/config.h.in b/src/config.h.in
index be29681..0f2ef38 100644
--- a/src/config.h.in
+++ b/src/config.h.in
@@ -22,11 +22,7 @@ Boolean macros such as HAVE_STDLIB_H and SUPPORT_PCRE2_8 should either be
defined (conventionally to 1) for TRUE, and not defined at all for FALSE. All
such macros are listed as a commented #undef in config.h.generic. Macros such
as MATCH_LIMIT, whose actual value is relevant, have defaults defined, but are
-surrounded by #ifndef/#endif lines so that the value can be overridden by -D.
-
-PCRE2 uses memmove() if HAVE_MEMMOVE is defined; otherwise it uses bcopy() if
-HAVE_BCOPY is defined. If your system has neither bcopy() nor memmove(), make
-sure both macros are undefined; an emulation function will then be used. */
+surrounded by #ifndef/#endif lines so that the value can be overridden by -D. */
/* By default, the \R escape sequence matches any Unicode line ending
character or sequence of characters. If BSR_ANYCRLF is defined (to any
@@ -44,9 +40,14 @@ sure both macros are undefined; an emulation function will then be used. */
assumes that all input strings are in EBCDIC. If you do not define this
macro, PCRE2 will assume input strings are ASCII or UTF-8/16/32 Unicode. It
is not possible to build a version of PCRE2 that supports both EBCDIC and
- UTF-8/16/32. */
+ ASCII or UTF-8/16/32. */
#undef EBCDIC
+/* To force an EBCDIC environment, define this macro to make the core PCRE2
+ library functions use EBCDIC codepage 1047, regardless of whether the
+ compiler supports it using C character literals. */
+#undef EBCDIC_IGNORING_COMPILER
+
/* In an EBCDIC environment, define this macro to any value to arrange for the
NL character to be 0x25 instead of the default 0x15. NL plays the role that
LF does in an ASCII/Unicode environment. */
@@ -58,9 +59,6 @@ sure both macros are undefined; an emulation function will then be used. */
/* Define this if your compiler supports __attribute__((uninitialized)) */
#undef HAVE_ATTRIBUTE_UNINITIALIZED
-/* Define to 1 if you have the `bcopy' function. */
-#undef HAVE_BCOPY
-
/* Define this if your compiler provides __assume() */
#undef HAVE_BUILTIN_ASSUME
@@ -94,9 +92,6 @@ sure both macros are undefined; an emulation function will then be used. */
/* Define to 1 if you have the `memfd_create' function. */
#undef HAVE_MEMFD_CREATE
-/* Define to 1 if you have the `memmove' function. */
-#undef HAVE_MEMMOVE
-
/* Define to 1 if you have the header file. */
#undef HAVE_MINIX_CONFIG_H
@@ -133,9 +128,6 @@ sure both macros are undefined; an emulation function will then be used. */
/* Define to 1 if you have the header file. */
#undef HAVE_STDLIB_H
-/* Define to 1 if you have the `strerror' function. */
-#undef HAVE_STRERROR
-
/* Define to 1 if you have the header file. */
#undef HAVE_STRINGS_H
@@ -266,10 +258,25 @@ sure both macros are undefined; an emulation function will then be used. */
allows for the buffering of "before" and "after" lines. */
#undef PCRE2GREP_MAX_BUFSIZE
+
+/* See PCRE2_EXP_DEFN; but this is applied to functions in the libpcre2-posix
+ library. */
+#undef PCRE2POSIX_EXP_DEFN
+
+/* Define to any value if linking libpcre2-posix dynamically. Ideally, if both
+ static and shared libraries are being built, then PCRE2POSIX_SHARED would
+ be defined only for the shared build. Indeed, this is a requirement on
+ Windows. However, when building with Autoconf and libtool, we compile the
+ sources once only to create both the static and shared library, so in this
+ case, PCRE2POSIX_SHARED should only be defined if the shared library is
+ being built, regardless of whether or not the static library is also being
+ built. */
+#undef PCRE2POSIX_SHARED
+
/* Define to any value to include debugging code. */
#undef PCRE2_DEBUG
-/* to make a symbol visible */
+/* Define to the annotation for making a symbol visible. */
#undef PCRE2_EXPORT
@@ -285,7 +292,12 @@ sure both macros are undefined; an emulation function will then be used. */
in the C sense, but which are internal to the library. */
#undef PCRE2_EXP_DEFN
-/* Define to any value if linking statically (TODO: make nice with Libtool) */
+/* Define to any value if linking statically. Ideally, if both static and
+ shared libraries are being built, then PCRE2_STATIC would be defined only
+ for the static build. Indeed, this is a requirement on Windows. With
+ Autoconf and libtool however, it is idiomatic to compile the sources once
+ to create both the static and shared library, so in this case, PCRE2_STATIC
+ should only be defined if no shared library is being built. */
#undef PCRE2_STATIC
/* Define to necessary symbol if this constant uses a non-standard name on
diff --git a/src/libpcre2-16.sym b/src/libpcre2-16.sym
new file mode 100644
index 0000000..fd5257d
--- /dev/null
+++ b/src/libpcre2-16.sym
@@ -0,0 +1,88 @@
+# First version of PCRE2 in which symbols were assigned versions.
+PCRE2_10.47 {
+ global:
+ pcre2_callout_enumerate_16;
+ pcre2_code_copy_16;
+ pcre2_code_copy_with_tables_16;
+ pcre2_code_free_16;
+ pcre2_compile_16;
+ pcre2_compile_context_copy_16;
+ pcre2_compile_context_create_16;
+ pcre2_compile_context_free_16;
+ pcre2_config_16;
+ pcre2_convert_context_copy_16;
+ pcre2_convert_context_create_16;
+ pcre2_convert_context_free_16;
+ pcre2_converted_pattern_free_16;
+ pcre2_dfa_match_16;
+ pcre2_general_context_copy_16;
+ pcre2_general_context_create_16;
+ pcre2_general_context_free_16;
+ pcre2_get_error_message_16;
+ pcre2_get_mark_16;
+ pcre2_get_match_data_heapframes_size_16;
+ pcre2_get_match_data_size_16;
+ pcre2_get_ovector_count_16;
+ pcre2_get_ovector_pointer_16;
+ pcre2_get_startchar_16;
+ pcre2_jit_compile_16;
+ pcre2_jit_free_unused_memory_16;
+ pcre2_jit_match_16;
+ pcre2_jit_stack_assign_16;
+ pcre2_jit_stack_create_16;
+ pcre2_jit_stack_free_16;
+ pcre2_maketables_16;
+ pcre2_maketables_free_16;
+ pcre2_match_16;
+ pcre2_match_context_copy_16;
+ pcre2_match_context_create_16;
+ pcre2_match_context_free_16;
+ pcre2_match_data_create_16;
+ pcre2_match_data_create_from_pattern_16;
+ pcre2_match_data_free_16;
+ pcre2_next_match_16;
+ pcre2_pattern_convert_16;
+ pcre2_pattern_info_16;
+ pcre2_serialize_decode_16;
+ pcre2_serialize_encode_16;
+ pcre2_serialize_free_16;
+ pcre2_serialize_get_number_of_codes_16;
+ pcre2_set_bsr_16;
+ pcre2_set_callout_16;
+ pcre2_set_character_tables_16;
+ pcre2_set_compile_extra_options_16;
+ pcre2_set_compile_recursion_guard_16;
+ pcre2_set_depth_limit_16;
+ pcre2_set_glob_escape_16;
+ pcre2_set_glob_separator_16;
+ pcre2_set_heap_limit_16;
+ pcre2_set_match_limit_16;
+ pcre2_set_max_pattern_compiled_length_16;
+ pcre2_set_max_pattern_length_16;
+ pcre2_set_max_varlookbehind_16;
+ pcre2_set_newline_16;
+ pcre2_set_offset_limit_16;
+ pcre2_set_optimize_16;
+ pcre2_set_parens_nest_limit_16;
+ pcre2_set_recursion_limit_16;
+ pcre2_set_recursion_memory_management_16;
+ pcre2_set_substitute_callout_16;
+ pcre2_set_substitute_case_callout_16;
+ pcre2_substitute_16;
+ pcre2_substring_copy_byname_16;
+ pcre2_substring_copy_bynumber_16;
+ pcre2_substring_free_16;
+ pcre2_substring_get_byname_16;
+ pcre2_substring_get_bynumber_16;
+ pcre2_substring_length_byname_16;
+ pcre2_substring_length_bynumber_16;
+ pcre2_substring_list_free_16;
+ pcre2_substring_list_get_16;
+ pcre2_substring_nametable_scan_16;
+ pcre2_substring_number_from_name_16;
+ local:
+ _fini;
+ _init;
+};
+
+# PCRE2_10.48 {} PCRE2_10.47;
diff --git a/src/libpcre2-32.sym b/src/libpcre2-32.sym
new file mode 100644
index 0000000..d4606c2
--- /dev/null
+++ b/src/libpcre2-32.sym
@@ -0,0 +1,88 @@
+# First version of PCRE2 in which symbols were assigned versions.
+PCRE2_10.47 {
+ global:
+ pcre2_callout_enumerate_32;
+ pcre2_code_copy_32;
+ pcre2_code_copy_with_tables_32;
+ pcre2_code_free_32;
+ pcre2_compile_32;
+ pcre2_compile_context_copy_32;
+ pcre2_compile_context_create_32;
+ pcre2_compile_context_free_32;
+ pcre2_config_32;
+ pcre2_convert_context_copy_32;
+ pcre2_convert_context_create_32;
+ pcre2_convert_context_free_32;
+ pcre2_converted_pattern_free_32;
+ pcre2_dfa_match_32;
+ pcre2_general_context_copy_32;
+ pcre2_general_context_create_32;
+ pcre2_general_context_free_32;
+ pcre2_get_error_message_32;
+ pcre2_get_mark_32;
+ pcre2_get_match_data_heapframes_size_32;
+ pcre2_get_match_data_size_32;
+ pcre2_get_ovector_count_32;
+ pcre2_get_ovector_pointer_32;
+ pcre2_get_startchar_32;
+ pcre2_jit_compile_32;
+ pcre2_jit_free_unused_memory_32;
+ pcre2_jit_match_32;
+ pcre2_jit_stack_assign_32;
+ pcre2_jit_stack_create_32;
+ pcre2_jit_stack_free_32;
+ pcre2_maketables_32;
+ pcre2_maketables_free_32;
+ pcre2_match_32;
+ pcre2_match_context_copy_32;
+ pcre2_match_context_create_32;
+ pcre2_match_context_free_32;
+ pcre2_match_data_create_32;
+ pcre2_match_data_create_from_pattern_32;
+ pcre2_match_data_free_32;
+ pcre2_next_match_32;
+ pcre2_pattern_convert_32;
+ pcre2_pattern_info_32;
+ pcre2_serialize_decode_32;
+ pcre2_serialize_encode_32;
+ pcre2_serialize_free_32;
+ pcre2_serialize_get_number_of_codes_32;
+ pcre2_set_bsr_32;
+ pcre2_set_callout_32;
+ pcre2_set_character_tables_32;
+ pcre2_set_compile_extra_options_32;
+ pcre2_set_compile_recursion_guard_32;
+ pcre2_set_depth_limit_32;
+ pcre2_set_glob_escape_32;
+ pcre2_set_glob_separator_32;
+ pcre2_set_heap_limit_32;
+ pcre2_set_match_limit_32;
+ pcre2_set_max_pattern_compiled_length_32;
+ pcre2_set_max_pattern_length_32;
+ pcre2_set_max_varlookbehind_32;
+ pcre2_set_newline_32;
+ pcre2_set_offset_limit_32;
+ pcre2_set_optimize_32;
+ pcre2_set_parens_nest_limit_32;
+ pcre2_set_recursion_limit_32;
+ pcre2_set_recursion_memory_management_32;
+ pcre2_set_substitute_callout_32;
+ pcre2_set_substitute_case_callout_32;
+ pcre2_substitute_32;
+ pcre2_substring_copy_byname_32;
+ pcre2_substring_copy_bynumber_32;
+ pcre2_substring_free_32;
+ pcre2_substring_get_byname_32;
+ pcre2_substring_get_bynumber_32;
+ pcre2_substring_length_byname_32;
+ pcre2_substring_length_bynumber_32;
+ pcre2_substring_list_free_32;
+ pcre2_substring_list_get_32;
+ pcre2_substring_nametable_scan_32;
+ pcre2_substring_number_from_name_32;
+ local:
+ _fini;
+ _init;
+};
+
+# PCRE2_10.48 {} PCRE2_10.47;
diff --git a/src/libpcre2-8.sym b/src/libpcre2-8.sym
new file mode 100644
index 0000000..368b4e4
--- /dev/null
+++ b/src/libpcre2-8.sym
@@ -0,0 +1,88 @@
+# First version of PCRE2 in which symbols were assigned versions.
+PCRE2_10.47 {
+ global:
+ pcre2_callout_enumerate_8;
+ pcre2_code_copy_8;
+ pcre2_code_copy_with_tables_8;
+ pcre2_code_free_8;
+ pcre2_compile_8;
+ pcre2_compile_context_copy_8;
+ pcre2_compile_context_create_8;
+ pcre2_compile_context_free_8;
+ pcre2_config_8;
+ pcre2_convert_context_copy_8;
+ pcre2_convert_context_create_8;
+ pcre2_convert_context_free_8;
+ pcre2_converted_pattern_free_8;
+ pcre2_dfa_match_8;
+ pcre2_general_context_copy_8;
+ pcre2_general_context_create_8;
+ pcre2_general_context_free_8;
+ pcre2_get_error_message_8;
+ pcre2_get_mark_8;
+ pcre2_get_match_data_heapframes_size_8;
+ pcre2_get_match_data_size_8;
+ pcre2_get_ovector_count_8;
+ pcre2_get_ovector_pointer_8;
+ pcre2_get_startchar_8;
+ pcre2_jit_compile_8;
+ pcre2_jit_free_unused_memory_8;
+ pcre2_jit_match_8;
+ pcre2_jit_stack_assign_8;
+ pcre2_jit_stack_create_8;
+ pcre2_jit_stack_free_8;
+ pcre2_maketables_8;
+ pcre2_maketables_free_8;
+ pcre2_match_8;
+ pcre2_match_context_copy_8;
+ pcre2_match_context_create_8;
+ pcre2_match_context_free_8;
+ pcre2_match_data_create_8;
+ pcre2_match_data_create_from_pattern_8;
+ pcre2_match_data_free_8;
+ pcre2_next_match_8;
+ pcre2_pattern_convert_8;
+ pcre2_pattern_info_8;
+ pcre2_serialize_decode_8;
+ pcre2_serialize_encode_8;
+ pcre2_serialize_free_8;
+ pcre2_serialize_get_number_of_codes_8;
+ pcre2_set_bsr_8;
+ pcre2_set_callout_8;
+ pcre2_set_character_tables_8;
+ pcre2_set_compile_extra_options_8;
+ pcre2_set_compile_recursion_guard_8;
+ pcre2_set_depth_limit_8;
+ pcre2_set_glob_escape_8;
+ pcre2_set_glob_separator_8;
+ pcre2_set_heap_limit_8;
+ pcre2_set_match_limit_8;
+ pcre2_set_max_pattern_compiled_length_8;
+ pcre2_set_max_pattern_length_8;
+ pcre2_set_max_varlookbehind_8;
+ pcre2_set_newline_8;
+ pcre2_set_offset_limit_8;
+ pcre2_set_optimize_8;
+ pcre2_set_parens_nest_limit_8;
+ pcre2_set_recursion_limit_8;
+ pcre2_set_recursion_memory_management_8;
+ pcre2_set_substitute_callout_8;
+ pcre2_set_substitute_case_callout_8;
+ pcre2_substitute_8;
+ pcre2_substring_copy_byname_8;
+ pcre2_substring_copy_bynumber_8;
+ pcre2_substring_free_8;
+ pcre2_substring_get_byname_8;
+ pcre2_substring_get_bynumber_8;
+ pcre2_substring_length_byname_8;
+ pcre2_substring_length_bynumber_8;
+ pcre2_substring_list_free_8;
+ pcre2_substring_list_get_8;
+ pcre2_substring_nametable_scan_8;
+ pcre2_substring_number_from_name_8;
+ local:
+ _fini;
+ _init;
+};
+
+# PCRE2_10.48 {} PCRE2_10.47;
diff --git a/src/libpcre2-posix.sym b/src/libpcre2-posix.sym
new file mode 100644
index 0000000..04a56cf
--- /dev/null
+++ b/src/libpcre2-posix.sym
@@ -0,0 +1,13 @@
+# First version of PCRE2 in which symbols were assigned versions.
+PCRE2_10.47 {
+ global:
+ pcre2_regcomp;
+ pcre2_regerror;
+ pcre2_regexec;
+ pcre2_regfree;
+ local:
+ _fini;
+ _init;
+};
+
+# PCRE2_10.48 {} PCRE2_10.47;
diff --git a/src/pcre2.h.generic b/src/pcre2.h.generic
index a6c739f..4034e5a 100644
--- a/src/pcre2.h.generic
+++ b/src/pcre2.h.generic
@@ -42,25 +42,21 @@ POSSIBILITY OF SUCH DAMAGE.
/* The current PCRE version information. */
#define PCRE2_MAJOR 10
-#define PCRE2_MINOR 46
+#define PCRE2_MINOR 47
#define PCRE2_PRERELEASE
-#define PCRE2_DATE 2025-08-27
+#define PCRE2_DATE 2025-10-21
-/* When an application links to a PCRE DLL in Windows, the symbols that are
+/* When an application links to a PCRE2 DLL in Windows, the symbols that are
imported have to be identified as such. When building PCRE2, the appropriate
-export setting is defined in pcre2_internal.h, which includes this file. So we
-don't change existing definitions of PCRE2_EXP_DECL. */
+export setting is defined in pcre2_internal.h, which includes this file. So, we
+don't change existing definitions of PCRE2_EXP_DECL.
-#if defined(_WIN32) && !defined(PCRE2_STATIC)
-# ifndef PCRE2_EXP_DECL
-# define PCRE2_EXP_DECL extern __declspec(dllimport)
-# endif
-#endif
-
-/* By default, we use the standard "extern" declarations. */
+By default, we use the standard "extern" declarations. */
#ifndef PCRE2_EXP_DECL
-# ifdef __cplusplus
+# if defined(_WIN32) && !defined(PCRE2_STATIC)
+# define PCRE2_EXP_DECL extern __declspec(dllimport)
+# elif defined __cplusplus
# define PCRE2_EXP_DECL extern "C"
# else
# define PCRE2_EXP_DECL extern
@@ -68,14 +64,15 @@ don't change existing definitions of PCRE2_EXP_DECL. */
#endif
/* When compiling with the MSVC compiler, it is sometimes necessary to include
-a "calling convention" before exported function names. (This is secondhand
-information; I know nothing about MSVC myself). For example, something like
+a "calling convention" before exported function names. For example:
void __cdecl function(....)
-might be needed. In order so make this easy, all the exported functions have
-PCRE2_CALL_CONVENTION just before their names. It is rarely needed; if not
-set, we ensure here that it has no effect. */
+might be needed. In order to make this easy, all the exported functions have
+PCRE2_CALL_CONVENTION just before their names.
+
+PCRE2 normally uses the platform's standard calling convention, so this should
+not be set unless you know you need it. */
#ifndef PCRE2_CALL_CONVENTION
#define PCRE2_CALL_CONVENTION
@@ -343,6 +340,10 @@ pcre2_pattern_convert(). */
#define PCRE2_ERROR_PERL_ECLASS_EMPTY_EXPR 214
#define PCRE2_ERROR_PERL_ECLASS_MISSING_CLOSE 215
#define PCRE2_ERROR_PERL_ECLASS_UNEXPECTED_CHAR 216
+#define PCRE2_ERROR_EXPECTED_CAPTURE_GROUP 217
+#define PCRE2_ERROR_MISSING_OPENING_PARENTHESIS 218
+#define PCRE2_ERROR_MISSING_NUMBER_TERMINATOR 219
+#define PCRE2_ERROR_NULL_ERROROFFSET 220
/* "Expected" matching error codes: no match and partial match. */
@@ -432,6 +433,11 @@ released, the numbers must not be changed. */
#define PCRE2_ERROR_JIT_UNSUPPORTED (-68)
#define PCRE2_ERROR_REPLACECASE (-69)
#define PCRE2_ERROR_TOOLARGEREPLACE (-70)
+#define PCRE2_ERROR_DIFFSUBSPATTERN (-71)
+#define PCRE2_ERROR_DIFFSUBSSUBJECT (-72)
+#define PCRE2_ERROR_DIFFSUBSOFFSET (-73)
+#define PCRE2_ERROR_DIFFSUBSOPTIONS (-74)
+#define PCRE2_ERROR_BAD_BACKSLASH_K (-75)
/* Request types for pcre2_pattern_info() */
@@ -484,6 +490,7 @@ released, the numbers must not be changed. */
#define PCRE2_CONFIG_NEVER_BACKSLASH_C 13
#define PCRE2_CONFIG_COMPILED_WIDTHS 14
#define PCRE2_CONFIG_TABLES_LENGTH 15
+#define PCRE2_CONFIG_EFFECTIVE_LINKSIZE 16
/* Optimization directives for pcre2_set_optimize().
For binary compatibility, only add to this list; do not renumber. */
@@ -743,14 +750,14 @@ PCRE2_EXP_DECL pcre2_match_data *PCRE2_CALL_CONVENTION \
PCRE2_EXP_DECL pcre2_match_data *PCRE2_CALL_CONVENTION \
pcre2_match_data_create_from_pattern(const pcre2_code *, \
pcre2_general_context *); \
+PCRE2_EXP_DECL void PCRE2_CALL_CONVENTION \
+ pcre2_match_data_free(pcre2_match_data *); \
PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \
pcre2_dfa_match(const pcre2_code *, PCRE2_SPTR, PCRE2_SIZE, PCRE2_SIZE, \
uint32_t, pcre2_match_data *, pcre2_match_context *, int *, PCRE2_SIZE); \
PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \
pcre2_match(const pcre2_code *, PCRE2_SPTR, PCRE2_SIZE, PCRE2_SIZE, \
uint32_t, pcre2_match_data *, pcre2_match_context *); \
-PCRE2_EXP_DECL void PCRE2_CALL_CONVENTION \
- pcre2_match_data_free(pcre2_match_data *); \
PCRE2_EXP_DECL PCRE2_SPTR PCRE2_CALL_CONVENTION \
pcre2_get_mark(pcre2_match_data *); \
PCRE2_EXP_DECL PCRE2_SIZE PCRE2_CALL_CONVENTION \
@@ -762,7 +769,9 @@ PCRE2_EXP_DECL uint32_t PCRE2_CALL_CONVENTION \
PCRE2_EXP_DECL PCRE2_SIZE *PCRE2_CALL_CONVENTION \
pcre2_get_ovector_pointer(pcre2_match_data *); \
PCRE2_EXP_DECL PCRE2_SIZE PCRE2_CALL_CONVENTION \
- pcre2_get_startchar(pcre2_match_data *);
+ pcre2_get_startchar(pcre2_match_data *); \
+PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \
+ pcre2_next_match(pcre2_match_data *, PCRE2_SIZE *, uint32_t *);
/* Convenience functions for handling matched substrings. */
@@ -942,6 +951,7 @@ pcre2_compile are called by application code. */
#define pcre2_match_data_create PCRE2_SUFFIX(pcre2_match_data_create_)
#define pcre2_match_data_create_from_pattern PCRE2_SUFFIX(pcre2_match_data_create_from_pattern_)
#define pcre2_match_data_free PCRE2_SUFFIX(pcre2_match_data_free_)
+#define pcre2_next_match PCRE2_SUFFIX(pcre2_next_match_)
#define pcre2_pattern_convert PCRE2_SUFFIX(pcre2_pattern_convert_)
#define pcre2_pattern_info PCRE2_SUFFIX(pcre2_pattern_info_)
#define pcre2_serialize_decode PCRE2_SUFFIX(pcre2_serialize_decode_)
diff --git a/src/pcre2.h.in b/src/pcre2.h.in
index ca3f0b4..0dffe65 100644
--- a/src/pcre2.h.in
+++ b/src/pcre2.h.in
@@ -46,21 +46,17 @@ POSSIBILITY OF SUCH DAMAGE.
#define PCRE2_PRERELEASE @PCRE2_PRERELEASE@
#define PCRE2_DATE @PCRE2_DATE@
-/* When an application links to a PCRE DLL in Windows, the symbols that are
+/* When an application links to a PCRE2 DLL in Windows, the symbols that are
imported have to be identified as such. When building PCRE2, the appropriate
-export setting is defined in pcre2_internal.h, which includes this file. So we
-don't change existing definitions of PCRE2_EXP_DECL. */
+export setting is defined in pcre2_internal.h, which includes this file. So, we
+don't change existing definitions of PCRE2_EXP_DECL.
-#if defined(_WIN32) && !defined(PCRE2_STATIC)
-# ifndef PCRE2_EXP_DECL
-# define PCRE2_EXP_DECL extern __declspec(dllimport)
-# endif
-#endif
-
-/* By default, we use the standard "extern" declarations. */
+By default, we use the standard "extern" declarations. */
#ifndef PCRE2_EXP_DECL
-# ifdef __cplusplus
+# if defined(_WIN32) && !defined(PCRE2_STATIC)
+# define PCRE2_EXP_DECL extern __declspec(dllimport)
+# elif defined __cplusplus
# define PCRE2_EXP_DECL extern "C"
# else
# define PCRE2_EXP_DECL extern
@@ -68,14 +64,15 @@ don't change existing definitions of PCRE2_EXP_DECL. */
#endif
/* When compiling with the MSVC compiler, it is sometimes necessary to include
-a "calling convention" before exported function names. (This is secondhand
-information; I know nothing about MSVC myself). For example, something like
+a "calling convention" before exported function names. For example:
void __cdecl function(....)
-might be needed. In order so make this easy, all the exported functions have
-PCRE2_CALL_CONVENTION just before their names. It is rarely needed; if not
-set, we ensure here that it has no effect. */
+might be needed. In order to make this easy, all the exported functions have
+PCRE2_CALL_CONVENTION just before their names.
+
+PCRE2 normally uses the platform's standard calling convention, so this should
+not be set unless you know you need it. */
#ifndef PCRE2_CALL_CONVENTION
#define PCRE2_CALL_CONVENTION
@@ -343,6 +340,10 @@ pcre2_pattern_convert(). */
#define PCRE2_ERROR_PERL_ECLASS_EMPTY_EXPR 214
#define PCRE2_ERROR_PERL_ECLASS_MISSING_CLOSE 215
#define PCRE2_ERROR_PERL_ECLASS_UNEXPECTED_CHAR 216
+#define PCRE2_ERROR_EXPECTED_CAPTURE_GROUP 217
+#define PCRE2_ERROR_MISSING_OPENING_PARENTHESIS 218
+#define PCRE2_ERROR_MISSING_NUMBER_TERMINATOR 219
+#define PCRE2_ERROR_NULL_ERROROFFSET 220
/* "Expected" matching error codes: no match and partial match. */
@@ -432,6 +433,11 @@ released, the numbers must not be changed. */
#define PCRE2_ERROR_JIT_UNSUPPORTED (-68)
#define PCRE2_ERROR_REPLACECASE (-69)
#define PCRE2_ERROR_TOOLARGEREPLACE (-70)
+#define PCRE2_ERROR_DIFFSUBSPATTERN (-71)
+#define PCRE2_ERROR_DIFFSUBSSUBJECT (-72)
+#define PCRE2_ERROR_DIFFSUBSOFFSET (-73)
+#define PCRE2_ERROR_DIFFSUBSOPTIONS (-74)
+#define PCRE2_ERROR_BAD_BACKSLASH_K (-75)
/* Request types for pcre2_pattern_info() */
@@ -484,6 +490,7 @@ released, the numbers must not be changed. */
#define PCRE2_CONFIG_NEVER_BACKSLASH_C 13
#define PCRE2_CONFIG_COMPILED_WIDTHS 14
#define PCRE2_CONFIG_TABLES_LENGTH 15
+#define PCRE2_CONFIG_EFFECTIVE_LINKSIZE 16
/* Optimization directives for pcre2_set_optimize().
For binary compatibility, only add to this list; do not renumber. */
@@ -743,14 +750,14 @@ PCRE2_EXP_DECL pcre2_match_data *PCRE2_CALL_CONVENTION \
PCRE2_EXP_DECL pcre2_match_data *PCRE2_CALL_CONVENTION \
pcre2_match_data_create_from_pattern(const pcre2_code *, \
pcre2_general_context *); \
+PCRE2_EXP_DECL void PCRE2_CALL_CONVENTION \
+ pcre2_match_data_free(pcre2_match_data *); \
PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \
pcre2_dfa_match(const pcre2_code *, PCRE2_SPTR, PCRE2_SIZE, PCRE2_SIZE, \
uint32_t, pcre2_match_data *, pcre2_match_context *, int *, PCRE2_SIZE); \
PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \
pcre2_match(const pcre2_code *, PCRE2_SPTR, PCRE2_SIZE, PCRE2_SIZE, \
uint32_t, pcre2_match_data *, pcre2_match_context *); \
-PCRE2_EXP_DECL void PCRE2_CALL_CONVENTION \
- pcre2_match_data_free(pcre2_match_data *); \
PCRE2_EXP_DECL PCRE2_SPTR PCRE2_CALL_CONVENTION \
pcre2_get_mark(pcre2_match_data *); \
PCRE2_EXP_DECL PCRE2_SIZE PCRE2_CALL_CONVENTION \
@@ -762,7 +769,9 @@ PCRE2_EXP_DECL uint32_t PCRE2_CALL_CONVENTION \
PCRE2_EXP_DECL PCRE2_SIZE *PCRE2_CALL_CONVENTION \
pcre2_get_ovector_pointer(pcre2_match_data *); \
PCRE2_EXP_DECL PCRE2_SIZE PCRE2_CALL_CONVENTION \
- pcre2_get_startchar(pcre2_match_data *);
+ pcre2_get_startchar(pcre2_match_data *); \
+PCRE2_EXP_DECL int PCRE2_CALL_CONVENTION \
+ pcre2_next_match(pcre2_match_data *, PCRE2_SIZE *, uint32_t *);
/* Convenience functions for handling matched substrings. */
@@ -942,6 +951,7 @@ pcre2_compile are called by application code. */
#define pcre2_match_data_create PCRE2_SUFFIX(pcre2_match_data_create_)
#define pcre2_match_data_create_from_pattern PCRE2_SUFFIX(pcre2_match_data_create_from_pattern_)
#define pcre2_match_data_free PCRE2_SUFFIX(pcre2_match_data_free_)
+#define pcre2_next_match PCRE2_SUFFIX(pcre2_next_match_)
#define pcre2_pattern_convert PCRE2_SUFFIX(pcre2_pattern_convert_)
#define pcre2_pattern_info PCRE2_SUFFIX(pcre2_pattern_info_)
#define pcre2_serialize_decode PCRE2_SUFFIX(pcre2_serialize_decode_)
diff --git a/src/pcre2_auto_possess.c b/src/pcre2_auto_possess.c
index 6d7f27b..4088032 100644
--- a/src/pcre2_auto_possess.c
+++ b/src/pcre2_auto_possess.c
@@ -38,16 +38,14 @@ POSSIBILITY OF SUCH DAMAGE.
-----------------------------------------------------------------------------
*/
+
/* This module contains functions that scan a compiled pattern and change
repeats into possessive repeats where possible. */
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
+#include "pcre2_internal.h"
-#include "pcre2_internal.h"
/* This macro represents the max size of list[] and that is used to keep
track of UCD info in several places, it should be kept on sync with the
@@ -264,8 +262,10 @@ switch(ptype)
if (c < *p) return !negated;
if (c == *p++) return negated;
}
+ /* LCOV_EXCL_START */
PCRE2_DEBUG_UNREACHABLE(); /* Control should never reach here */
break;
+ /* LCOV_EXCL_STOP */
/* Haven't yet thought these through. */
@@ -806,21 +806,21 @@ for(;;)
case OP_NOT_DIGIT:
invert_bits = TRUE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_DIGIT:
set2 = (const uint8_t *)(cb->cbits + cbit_digit);
break;
case OP_NOT_WHITESPACE:
invert_bits = TRUE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_WHITESPACE:
set2 = (const uint8_t *)(cb->cbits + cbit_space);
break;
case OP_NOT_WORDCHAR:
invert_bits = TRUE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_WORDCHAR:
set2 = (const uint8_t *)(cb->cbits + cbit_word);
break;
@@ -1103,7 +1103,7 @@ for(;;)
case OP_NCLASS:
if (chr > 255) return FALSE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_CLASS:
if (chr > 255) break;
@@ -1141,8 +1141,10 @@ for(;;)
if (list[1] == 0) return TRUE;
}
+/* LCOV_EXCL_START */
PCRE2_DEBUG_UNREACHABLE(); /* Control should never reach here */
return FALSE; /* Avoid compiler warnings */
+/* LCOV_EXCL_STOP */
}
@@ -1181,11 +1183,13 @@ for (;;)
{
c = *code;
+ /* LCOV_EXCL_START */
if (c >= OP_TABLE_LENGTH)
{
PCRE2_DEBUG_UNREACHABLE();
return -1; /* Something gone wrong */
}
+ /* LCOV_EXCL_STOP */
if (c >= OP_STAR && c <= OP_TYPEPOSUPTO)
{
diff --git a/src/pcre2_chartables.c.dist b/src/pcre2_chartables.c.dist
index 7362c3f..29f9200 100644
--- a/src/pcre2_chartables.c.dist
+++ b/src/pcre2_chartables.c.dist
@@ -19,10 +19,6 @@ PCRE2 is configured with --enable-rebuild-chartables. However, you can run
pcre2_dftables manually with the -L option to build tables using the LC_ALL
locale. */
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-
#include "pcre2_internal.h"
const uint8_t PRIV(default_tables)[] = {
diff --git a/src/pcre2_chartables.c.ebcdic-1047-nl15 b/src/pcre2_chartables.c.ebcdic-1047-nl15
new file mode 100644
index 0000000..071a8f4
--- /dev/null
+++ b/src/pcre2_chartables.c.ebcdic-1047-nl15
@@ -0,0 +1,196 @@
+/*************************************************
+* Perl-Compatible Regular Expressions *
+*************************************************/
+
+/* This file was automatically written by the pcre2_dftables auxiliary
+program. It contains character tables that are used when no external
+tables are passed to PCRE2 by the application that calls it. The tables
+are used only for characters whose code values are less than 256, and
+only relevant if not in UCP mode. */
+
+/* This set of tables was written in the EBCDIC 1047 (NL 0x15) locale. */
+
+/* The pcre2_ftables program (which is distributed with PCRE2) can be used
+to build alternative versions of this file. This is necessary if you are
+running in an EBCDIC environment, or if you want to default to a different
+encoding, for example ISO-8859-1. When pcre2_dftables is run, it creates
+these tables in the "C" locale by default. This happens automatically if
+PCRE2 is configured with --enable-rebuild-chartables. However, you can run
+pcre2_dftables manually with the -L option to build tables using the LC_ALL
+locale. */
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "pcre2_internal.h"
+
+const uint8_t PRIV(default_tables)[] = {
+
+/* This table is a lower casing table. */
+
+ 0, 1, 2, 3, 4, 5, 6, 7,
+ 8, 9, 10, 11, 12, 13, 14, 15,
+ 16, 17, 18, 19, 20, 21, 22, 23,
+ 24, 25, 26, 27, 28, 29, 30, 31,
+ 32, 33, 34, 35, 36, 37, 38, 39,
+ 40, 41, 42, 43, 44, 45, 46, 47,
+ 48, 49, 50, 51, 52, 53, 54, 55,
+ 56, 57, 58, 59, 60, 61, 62, 63,
+ 64, 65, 66, 67, 68, 69, 70, 71,
+ 72, 73, 74, 75, 76, 77, 78, 79,
+ 80, 81, 82, 83, 84, 85, 86, 87,
+ 88, 89, 90, 91, 92, 93, 94, 95,
+ 96, 97, 98, 99,100,101,102,103,
+ 104,105,106,107,108,109,110,111,
+ 112,113,114,115,116,117,118,119,
+ 120,121,122,123,124,125,126,127,
+ 128,129,130,131,132,133,134,135,
+ 136,137,138,139,140,141,142,143,
+ 144,145,146,147,148,149,150,151,
+ 152,153,154,155,156,157,158,159,
+ 160,161,162,163,164,165,166,167,
+ 168,169,170,171,172,173,174,175,
+ 176,177,178,179,180,181,182,183,
+ 184,185,186,187,188,189,190,191,
+ 192,129,130,131,132,133,134,135,
+ 136,137,202,203,204,205,206,207,
+ 208,145,146,147,148,149,150,151,
+ 152,153,218,219,220,221,222,223,
+ 224,225,162,163,164,165,166,167,
+ 168,169,234,235,236,237,238,239,
+ 240,241,242,243,244,245,246,247,
+ 248,249,250,251,252,253,254,255,
+
+/* This table is a case flipping table. */
+
+ 0, 1, 2, 3, 4, 5, 6, 7,
+ 8, 9, 10, 11, 12, 13, 14, 15,
+ 16, 17, 18, 19, 20, 21, 22, 23,
+ 24, 25, 26, 27, 28, 29, 30, 31,
+ 32, 33, 34, 35, 36, 37, 38, 39,
+ 40, 41, 42, 43, 44, 45, 46, 47,
+ 48, 49, 50, 51, 52, 53, 54, 55,
+ 56, 57, 58, 59, 60, 61, 62, 63,
+ 64, 65, 66, 67, 68, 69, 70, 71,
+ 72, 73, 74, 75, 76, 77, 78, 79,
+ 80, 81, 82, 83, 84, 85, 86, 87,
+ 88, 89, 90, 91, 92, 93, 94, 95,
+ 96, 97, 98, 99,100,101,102,103,
+ 104,105,106,107,108,109,110,111,
+ 112,113,114,115,116,117,118,119,
+ 120,121,122,123,124,125,126,127,
+ 128,193,194,195,196,197,198,199,
+ 200,201,138,139,140,141,142,143,
+ 144,209,210,211,212,213,214,215,
+ 216,217,154,155,156,157,158,159,
+ 160,161,226,227,228,229,230,231,
+ 232,233,170,171,172,173,174,175,
+ 176,177,178,179,180,181,182,183,
+ 184,185,186,187,188,189,190,191,
+ 192,129,130,131,132,133,134,135,
+ 136,137,202,203,204,205,206,207,
+ 208,145,146,147,148,149,150,151,
+ 152,153,218,219,220,221,222,223,
+ 224,225,162,163,164,165,166,167,
+ 168,169,234,235,236,237,238,239,
+ 240,241,242,243,244,245,246,247,
+ 248,249,250,251,252,253,254,255,
+
+/* This table contains bit maps for various character classes. Each map is 32
+bytes long and the bits run from the least significant end of each byte. The
+classes that have their own maps are: space, xdigit, digit, upper, lower, word,
+graph, print, punct, and cntrl. Other classes are built from combinations. */
+
+ 0x20,0x38,0x20,0x00,0x00,0x00,0x00,0x00, /* space */
+ 0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* xdigit */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x7e,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x7e,0x00,0x00,0x00,0x00,0x00,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* digit */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* upper */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0x00,0x00,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* lower */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* word */
+ 0x00,0x00,0x00,0x00,0x00,0x20,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* graph */
+ 0x00,0xf8,0x01,0xfc,0x03,0xf8,0x00,0xfe,
+ 0xfe,0x03,0xfe,0x03,0xfe,0x23,0x00,0x20,
+ 0xff,0x03,0xff,0x03,0xfd,0x03,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* print */
+ 0x01,0xf8,0x01,0xfc,0x03,0xf8,0x00,0xfe,
+ 0xfe,0x03,0xfe,0x03,0xfe,0x23,0x00,0x20,
+ 0xff,0x03,0xff,0x03,0xfd,0x03,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* punct */
+ 0x00,0xf8,0x01,0xfc,0x03,0xf8,0x00,0xfe,
+ 0x00,0x00,0x00,0x00,0x02,0x20,0x00,0x20,
+ 0x01,0x00,0x01,0x00,0x01,0x00,0x00,0x00,
+
+ 0xaf,0xf8,0x6f,0xf3,0xc0,0xe0,0x84,0xb0, /* cntrl */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+
+/* This table identifies various classes of character by individual bits:
+ 0x01 white space character
+ 0x02 letter
+ 0x04 lower case letter
+ 0x08 decimal digit
+ 0x10 word (alphanumeric or '_')
+*/
+
+ 0x00,0x00,0x00,0x00,0x00,0x01,0x00,0x00, /* 0- 7 */
+ 0x00,0x00,0x00,0x01,0x01,0x01,0x00,0x00, /* 8- 15 */
+ 0x00,0x00,0x00,0x00,0x00,0x01,0x00,0x00, /* 16- 23 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 24- 31 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* - ' */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* ( - / */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 0 - 7 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 8 - ? */
+ 0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* @ - G */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* H - O */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* P - W */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* X - _ */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* ` - g */
+ 0x00,0x00,0x00,0x00,0x00,0x10,0x00,0x00, /* h - o */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* p - w */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* x -127 */
+ 0x00,0x16,0x16,0x16,0x16,0x16,0x16,0x16, /* 128-135 */
+ 0x16,0x16,0x00,0x00,0x00,0x00,0x00,0x00, /* 136-143 */
+ 0x00,0x16,0x16,0x16,0x16,0x16,0x16,0x16, /* 144-151 */
+ 0x16,0x16,0x00,0x00,0x00,0x00,0x00,0x00, /* 152-159 */
+ 0x00,0x00,0x16,0x16,0x16,0x16,0x16,0x16, /* 160-167 */
+ 0x16,0x16,0x00,0x00,0x00,0x00,0x00,0x00, /* 168-175 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 176-183 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 184-191 */
+ 0x00,0x12,0x12,0x12,0x12,0x12,0x12,0x12, /* 192-199 */
+ 0x12,0x12,0x00,0x00,0x00,0x00,0x00,0x00, /* 200-207 */
+ 0x00,0x12,0x12,0x12,0x12,0x12,0x12,0x12, /* 208-215 */
+ 0x12,0x12,0x00,0x00,0x00,0x00,0x00,0x00, /* 216-223 */
+ 0x00,0x00,0x12,0x12,0x12,0x12,0x12,0x12, /* 224-231 */
+ 0x12,0x12,0x00,0x00,0x00,0x00,0x00,0x00, /* 232-239 */
+ 0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18, /* 240-247 */
+ 0x18,0x18,0x00,0x00,0x00,0x00,0x00,0x00};/* 248-255 */
+
+/* End of pcre2_chartables.c */
diff --git a/src/pcre2_chartables.c.ebcdic-1047-nl25 b/src/pcre2_chartables.c.ebcdic-1047-nl25
new file mode 100644
index 0000000..99ba81b
--- /dev/null
+++ b/src/pcre2_chartables.c.ebcdic-1047-nl25
@@ -0,0 +1,196 @@
+/*************************************************
+* Perl-Compatible Regular Expressions *
+*************************************************/
+
+/* This file was automatically written by the pcre2_dftables auxiliary
+program. It contains character tables that are used when no external
+tables are passed to PCRE2 by the application that calls it. The tables
+are used only for characters whose code values are less than 256, and
+only relevant if not in UCP mode. */
+
+/* This set of tables was written in the EBCDIC 1047 (NL 0x25) locale. */
+
+/* The pcre2_ftables program (which is distributed with PCRE2) can be used
+to build alternative versions of this file. This is necessary if you are
+running in an EBCDIC environment, or if you want to default to a different
+encoding, for example ISO-8859-1. When pcre2_dftables is run, it creates
+these tables in the "C" locale by default. This happens automatically if
+PCRE2 is configured with --enable-rebuild-chartables. However, you can run
+pcre2_dftables manually with the -L option to build tables using the LC_ALL
+locale. */
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "pcre2_internal.h"
+
+const uint8_t PRIV(default_tables)[] = {
+
+/* This table is a lower casing table. */
+
+ 0, 1, 2, 3, 4, 5, 6, 7,
+ 8, 9, 10, 11, 12, 13, 14, 15,
+ 16, 17, 18, 19, 20, 21, 22, 23,
+ 24, 25, 26, 27, 28, 29, 30, 31,
+ 32, 33, 34, 35, 36, 37, 38, 39,
+ 40, 41, 42, 43, 44, 45, 46, 47,
+ 48, 49, 50, 51, 52, 53, 54, 55,
+ 56, 57, 58, 59, 60, 61, 62, 63,
+ 64, 65, 66, 67, 68, 69, 70, 71,
+ 72, 73, 74, 75, 76, 77, 78, 79,
+ 80, 81, 82, 83, 84, 85, 86, 87,
+ 88, 89, 90, 91, 92, 93, 94, 95,
+ 96, 97, 98, 99,100,101,102,103,
+ 104,105,106,107,108,109,110,111,
+ 112,113,114,115,116,117,118,119,
+ 120,121,122,123,124,125,126,127,
+ 128,129,130,131,132,133,134,135,
+ 136,137,138,139,140,141,142,143,
+ 144,145,146,147,148,149,150,151,
+ 152,153,154,155,156,157,158,159,
+ 160,161,162,163,164,165,166,167,
+ 168,169,170,171,172,173,174,175,
+ 176,177,178,179,180,181,182,183,
+ 184,185,186,187,188,189,190,191,
+ 192,129,130,131,132,133,134,135,
+ 136,137,202,203,204,205,206,207,
+ 208,145,146,147,148,149,150,151,
+ 152,153,218,219,220,221,222,223,
+ 224,225,162,163,164,165,166,167,
+ 168,169,234,235,236,237,238,239,
+ 240,241,242,243,244,245,246,247,
+ 248,249,250,251,252,253,254,255,
+
+/* This table is a case flipping table. */
+
+ 0, 1, 2, 3, 4, 5, 6, 7,
+ 8, 9, 10, 11, 12, 13, 14, 15,
+ 16, 17, 18, 19, 20, 21, 22, 23,
+ 24, 25, 26, 27, 28, 29, 30, 31,
+ 32, 33, 34, 35, 36, 37, 38, 39,
+ 40, 41, 42, 43, 44, 45, 46, 47,
+ 48, 49, 50, 51, 52, 53, 54, 55,
+ 56, 57, 58, 59, 60, 61, 62, 63,
+ 64, 65, 66, 67, 68, 69, 70, 71,
+ 72, 73, 74, 75, 76, 77, 78, 79,
+ 80, 81, 82, 83, 84, 85, 86, 87,
+ 88, 89, 90, 91, 92, 93, 94, 95,
+ 96, 97, 98, 99,100,101,102,103,
+ 104,105,106,107,108,109,110,111,
+ 112,113,114,115,116,117,118,119,
+ 120,121,122,123,124,125,126,127,
+ 128,193,194,195,196,197,198,199,
+ 200,201,138,139,140,141,142,143,
+ 144,209,210,211,212,213,214,215,
+ 216,217,154,155,156,157,158,159,
+ 160,161,226,227,228,229,230,231,
+ 232,233,170,171,172,173,174,175,
+ 176,177,178,179,180,181,182,183,
+ 184,185,186,187,188,189,190,191,
+ 192,129,130,131,132,133,134,135,
+ 136,137,202,203,204,205,206,207,
+ 208,145,146,147,148,149,150,151,
+ 152,153,218,219,220,221,222,223,
+ 224,225,162,163,164,165,166,167,
+ 168,169,234,235,236,237,238,239,
+ 240,241,242,243,244,245,246,247,
+ 248,249,250,251,252,253,254,255,
+
+/* This table contains bit maps for various character classes. Each map is 32
+bytes long and the bits run from the least significant end of each byte. The
+classes that have their own maps are: space, xdigit, digit, upper, lower, word,
+graph, print, punct, and cntrl. Other classes are built from combinations. */
+
+ 0x20,0x38,0x00,0x00,0x20,0x00,0x00,0x00, /* space */
+ 0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* xdigit */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x7e,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x7e,0x00,0x00,0x00,0x00,0x00,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* digit */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* upper */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0x00,0x00,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* lower */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* word */
+ 0x00,0x00,0x00,0x00,0x00,0x20,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0x00,0x00,
+ 0xfe,0x03,0xfe,0x03,0xfc,0x03,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* graph */
+ 0x00,0xf8,0x01,0xfc,0x03,0xf8,0x00,0xfe,
+ 0xfe,0x03,0xfe,0x03,0xfe,0x23,0x00,0x20,
+ 0xff,0x03,0xff,0x03,0xfd,0x03,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* print */
+ 0x01,0xf8,0x01,0xfc,0x03,0xf8,0x00,0xfe,
+ 0xfe,0x03,0xfe,0x03,0xfe,0x23,0x00,0x20,
+ 0xff,0x03,0xff,0x03,0xfd,0x03,0xff,0x03,
+
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* punct */
+ 0x00,0xf8,0x01,0xfc,0x03,0xf8,0x00,0xfe,
+ 0x00,0x00,0x00,0x00,0x02,0x20,0x00,0x20,
+ 0x01,0x00,0x01,0x00,0x01,0x00,0x00,0x00,
+
+ 0xaf,0xf8,0x4f,0xf3,0xe0,0xe0,0x84,0xb0, /* cntrl */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
+
+/* This table identifies various classes of character by individual bits:
+ 0x01 white space character
+ 0x02 letter
+ 0x04 lower case letter
+ 0x08 decimal digit
+ 0x10 word (alphanumeric or '_')
+*/
+
+ 0x00,0x00,0x00,0x00,0x00,0x01,0x00,0x00, /* 0- 7 */
+ 0x00,0x00,0x00,0x01,0x01,0x01,0x00,0x00, /* 8- 15 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 16- 23 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 24- 31 */
+ 0x00,0x00,0x00,0x00,0x00,0x01,0x00,0x00, /* - ' */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* ( - / */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 0 - 7 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 8 - ? */
+ 0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* @ - G */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* H - O */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* P - W */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* X - _ */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* ` - g */
+ 0x00,0x00,0x00,0x00,0x00,0x10,0x00,0x00, /* h - o */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* p - w */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* x -127 */
+ 0x00,0x16,0x16,0x16,0x16,0x16,0x16,0x16, /* 128-135 */
+ 0x16,0x16,0x00,0x00,0x00,0x00,0x00,0x00, /* 136-143 */
+ 0x00,0x16,0x16,0x16,0x16,0x16,0x16,0x16, /* 144-151 */
+ 0x16,0x16,0x00,0x00,0x00,0x00,0x00,0x00, /* 152-159 */
+ 0x00,0x00,0x16,0x16,0x16,0x16,0x16,0x16, /* 160-167 */
+ 0x16,0x16,0x00,0x00,0x00,0x00,0x00,0x00, /* 168-175 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 176-183 */
+ 0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, /* 184-191 */
+ 0x00,0x12,0x12,0x12,0x12,0x12,0x12,0x12, /* 192-199 */
+ 0x12,0x12,0x00,0x00,0x00,0x00,0x00,0x00, /* 200-207 */
+ 0x00,0x12,0x12,0x12,0x12,0x12,0x12,0x12, /* 208-215 */
+ 0x12,0x12,0x00,0x00,0x00,0x00,0x00,0x00, /* 216-223 */
+ 0x00,0x00,0x12,0x12,0x12,0x12,0x12,0x12, /* 224-231 */
+ 0x12,0x12,0x00,0x00,0x00,0x00,0x00,0x00, /* 232-239 */
+ 0x18,0x18,0x18,0x18,0x18,0x18,0x18,0x18, /* 240-247 */
+ 0x18,0x18,0x00,0x00,0x00,0x00,0x00,0x00};/* 248-255 */
+
+/* End of pcre2_chartables.c */
diff --git a/src/pcre2_chkdint.c b/src/pcre2_chkdint.c
index 7083023..7aeb640 100644
--- a/src/pcre2_chkdint.c
+++ b/src/pcre2_chkdint.c
@@ -37,16 +37,16 @@ POSSIBILITY OF SUCH DAMAGE.
-----------------------------------------------------------------------------
*/
+
/* This file contains functions to implement checked integer operation */
-#ifndef PCRE2_PCRE2TEST
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
+#ifndef PCRE2_PCRE2TEST
#include "pcre2_internal.h"
#endif
+
+
/*************************************************
* Checked Integer Multiplication *
*************************************************/
diff --git a/src/pcre2_compile.c b/src/pcre2_compile.c
index 0ffac89..f126e41 100644
--- a/src/pcre2_compile.c
+++ b/src/pcre2_compile.c
@@ -39,16 +39,14 @@ POSSIBILITY OF SUCH DAMAGE.
*/
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
+#include "pcre2_compile.h"
+
+
#define NLBLOCK cb /* Block containing newline information */
#define PSSTART start_pattern /* Field containing processed string start */
#define PSEND end_pattern /* Field containing processed string end */
-#include "pcre2_compile.h"
-
/* In rare error cases debugging might require calling pcre2_printint(). */
#if 0
@@ -57,7 +55,15 @@ POSSIBILITY OF SUCH DAMAGE.
#else
#define PRINTABLE(c) ((c) >= 32 && (c) < 127)
#endif
-#include "pcre2_printint.c"
+#define CHAR_OUTPUT(c) (c)
+#define CHAR_OUTPUT_HEX(c) (c)
+#define CHAR_INPUT(c) (c)
+#define CHAR_INPUT_HEX(c) (c)
+#include "pcre2_printint_inc.h"
+#undef PRINTABLE
+#undef CHAR_OUTPUT
+#undef CHAR_OUTPUT_HEX
+#undef CHAR_INPUT
#define DEBUG_CALL_PRINTINT
#endif
@@ -84,30 +90,6 @@ by defining macros in order to minimize #if usage. */
#endif
#endif
-/* Macros to store and retrieve a PCRE2_SIZE value in the parsed pattern, which
-consists of uint32_t elements. Assume that if uint32_t can't hold it, two of
-them will be able to (i.e. assume a 64-bit world). */
-
-#if PCRE2_SIZE_MAX <= UINT32_MAX
-#define PUTOFFSET(s,p) *p++ = s
-#define GETOFFSET(s,p) s = *p++
-#define GETPLUSOFFSET(s,p) s = *(++p)
-#define READPLUSOFFSET(s,p) s = p[1]
-#define SKIPOFFSET(p) p++
-#define SIZEOFFSET 1
-#else
-#define PUTOFFSET(s,p) \
- { *p++ = (uint32_t)(s >> 32); *p++ = (uint32_t)(s & 0xffffffff); }
-#define GETOFFSET(s,p) \
- { s = ((PCRE2_SIZE)p[0] << 32) | (PCRE2_SIZE)p[1]; p += 2; }
-#define GETPLUSOFFSET(s,p) \
- { s = ((PCRE2_SIZE)p[1] << 32) | (PCRE2_SIZE)p[2]; p += 2; }
-#define READPLUSOFFSET(s,p) \
- { s = ((PCRE2_SIZE)p[1] << 32) | (PCRE2_SIZE)p[2]; }
-#define SKIPOFFSET(p) p += 2
-#define SIZEOFFSET 2
-#endif
-
/* Function definitions to allow mutual recursion */
static int
@@ -216,8 +198,8 @@ static unsigned char meta_extra_lengths[] = {
3, /* META_COND_VERSION */
SIZEOFFSET, /* META_OFFSET */
0, /* META_SCS */
- 1, /* META_SCS_NAME */
- 1, /* META_SCS_NUMBER */
+ 1, /* META_CAPTURE_NAME */
+ 1, /* META_CAPTURE_NUMBER */
0, /* META_DOLLAR */
0, /* META_DOT */
0, /* META_ESCAPE - one more for ESC_P and ESC_p */
@@ -400,10 +382,10 @@ static const short int escapes[] = {
/* 4 */ 0, /* 5 */ 0,
/* 6 */ 0, /* 7 */ 0,
/* 8 */ 0, /* 9 */ 0,
- /* : */ CHAR_COLON, /* ; */ CHAR_SEMICOLON,
- /* < */ CHAR_LESS_THAN_SIGN, /* = */ CHAR_EQUALS_SIGN,
- /* > */ CHAR_GREATER_THAN_SIGN, /* ? */ CHAR_QUESTION_MARK,
- /* @ */ CHAR_COMMERCIAL_AT, /* A */ -ESC_A,
+ /* : */ ESCAPES_FIRST+0x0a, /* ; */ ESCAPES_FIRST+0x0b,
+ /* < */ ESCAPES_FIRST+0x0c, /* = */ ESCAPES_FIRST+0x0d,
+ /* > */ ESCAPES_FIRST+0x0e, /* ? */ ESCAPES_FIRST+0x0f,
+ /* @ */ ESCAPES_FIRST+0x10, /* A */ -ESC_A,
/* B */ -ESC_B, /* C */ -ESC_C,
/* D */ -ESC_D, /* E */ -ESC_E,
/* F */ 0, /* G */ -ESC_G,
@@ -416,10 +398,10 @@ static const short int escapes[] = {
/* T */ 0, /* U */ 0,
/* V */ -ESC_V, /* W */ -ESC_W,
/* X */ -ESC_X, /* Y */ 0,
- /* Z */ -ESC_Z, /* [ */ CHAR_LEFT_SQUARE_BRACKET,
- /* \ */ CHAR_BACKSLASH, /* ] */ CHAR_RIGHT_SQUARE_BRACKET,
- /* ^ */ CHAR_CIRCUMFLEX_ACCENT, /* _ */ CHAR_UNDERSCORE,
- /* ` */ CHAR_GRAVE_ACCENT, /* a */ CHAR_BEL,
+ /* Z */ -ESC_Z, /* [ */ ESCAPES_FIRST+0x2b,
+ /* \ */ ESCAPES_FIRST+0x2c, /* ] */ ESCAPES_FIRST+0x2d,
+ /* ^ */ ESCAPES_FIRST+0x2e, /* _ */ ESCAPES_FIRST+0x2f,
+ /* ` */ ESCAPES_FIRST+0x30, /* a */ CHAR_BEL,
/* b */ -ESC_b, /* c */ 0,
/* d */ -ESC_d, /* e */ CHAR_ESC,
/* f */ CHAR_FF, /* g */ 0,
@@ -438,43 +420,94 @@ static const short int escapes[] = {
#else
/* This is the "abnormal" table for EBCDIC systems without UTF-8 support.
-It runs from 'a' to '9'. For some minimal testing of EBCDIC features, the code
-is sometimes compiled on an ASCII system. In this case, we must not use CHAR_a
-because it is defined as 'a', which of course picks up the ASCII value. */
+It runs from 'a' to '9'. Our EBCDIC support can be provided via the compiler,
+which can interpret character literals like 'a' or '[' in an EBCDIC codepage;
+in this case, there is wide variance between codepages on the interpretation of
+characters between the letters ('[' and '{' and so on are placed in all sorts of
+different positions in the table). Thankfully however, all EBCDIC codepages
+place the letters and digits in the same location, so we hardcode that here.
+Our EBCDIC support can also be provided via numeric literals instead of
+character literals, so either way, 'CHAR_a' will be 0x81 when PCRE2 is compiled
+in EBCDIC mode. */
-#if 'a' == 0x81 /* Check for a real EBCDIC environment */
#define ESCAPES_FIRST CHAR_a
#define ESCAPES_LAST CHAR_9
#define UPPER_CASE(c) (c+64)
-#else /* Testing in an ASCII environment */
-#define ESCAPES_FIRST ((unsigned char)'\x81') /* EBCDIC 'a' */
-#define ESCAPES_LAST ((unsigned char)'\xf9') /* EBCDIC '9' */
-#define UPPER_CASE(c) (c-32)
-#endif
static const short int escapes[] = {
-/* 80 */ CHAR_BEL, -ESC_b, 0, -ESC_d, CHAR_ESC, CHAR_FF, 0,
-/* 88 */ -ESC_h, 0, 0, '{', 0, 0, 0, 0,
-/* 90 */ 0, 0, -ESC_k, 0, 0, CHAR_LF, 0, -ESC_p,
-/* 98 */ 0, CHAR_CR, 0, '}', 0, 0, 0, 0,
-/* A0 */ 0, '~', -ESC_s, CHAR_HT, 0, -ESC_v, -ESC_w, 0,
-/* A8 */ 0, -ESC_z, 0, 0, 0, '[', 0, 0,
-/* B0 */ 0, 0, 0, 0, 0, 0, 0, 0,
-/* B8 */ 0, 0, 0, 0, 0, ']', '=', '-',
-/* C0 */ '{', -ESC_A, -ESC_B, -ESC_C, -ESC_D, -ESC_E, 0, -ESC_G,
-/* C8 */ -ESC_H, 0, 0, 0, 0, 0, 0, 0,
-/* D0 */ '}', 0, -ESC_K, 0, 0, -ESC_N, 0, -ESC_P,
-/* D8 */ -ESC_Q, -ESC_R, 0, 0, 0, 0, 0, 0,
-/* E0 */ '\\', 0, -ESC_S, 0, 0, -ESC_V, -ESC_W, -ESC_X,
-/* E8 */ 0, -ESC_Z, 0, 0, 0, 0, 0, 0,
-/* F0 */ 0, 0, 0, 0, 0, 0, 0, 0,
-/* F8 */ 0, 0
+ /* 0x81 a */ CHAR_BEL, /* 0x82 b */ -ESC_b,
+ /* 0x83 c */ 0, /* 0x84 d */ -ESC_d,
+ /* 0x85 e */ CHAR_ESC, /* 0x86 f */ CHAR_FF,
+ /* 0x87 g */ 0, /* 0x88 h */ -ESC_h,
+ /* 0x89 i */ 0, /* 0x8a */ ESCAPES_FIRST+0x09,
+ /* 0x8b */ ESCAPES_FIRST+0x0a, /* 0x8c */ ESCAPES_FIRST+0x0b,
+ /* 0x8d */ ESCAPES_FIRST+0x0c, /* 0x8e */ ESCAPES_FIRST+0x0d,
+ /* 0x8f */ ESCAPES_FIRST+0x0e, /* 0x90 */ ESCAPES_FIRST+0x0f,
+ /* 0x91 j */ 0, /* 0x92 k */ -ESC_k,
+ /* 0x93 l */ 0, /* 0x94 m */ 0,
+ /* 0x95 n */ CHAR_LF, /* 0x96 o */ 0,
+ /* 0x97 p */ -ESC_p, /* 0x98 q */ 0,
+ /* 0x99 r */ CHAR_CR, /* 0x9a */ ESCAPES_FIRST+0x19,
+ /* 0x9b */ ESCAPES_FIRST+0x1a, /* 0x9c */ ESCAPES_FIRST+0x1b,
+ /* 0x9d */ ESCAPES_FIRST+0x1c, /* 0x9e */ ESCAPES_FIRST+0x1d,
+ /* 0x9f */ ESCAPES_FIRST+0x1e, /* 0xa0 */ ESCAPES_FIRST+0x1f,
+ /* 0xa1 */ ESCAPES_FIRST+0x20, /* 0xa2 s */ -ESC_s,
+ /* 0xa3 t */ CHAR_HT, /* 0xa4 u */ 0,
+ /* 0xa5 v */ -ESC_v, /* 0xa6 w */ -ESC_w,
+ /* 0xa7 x */ 0, /* 0xa8 y */ 0,
+ /* 0xa9 z */ -ESC_z, /* 0xaa */ ESCAPES_FIRST+0x29,
+ /* 0xab */ ESCAPES_FIRST+0x2a, /* 0xac */ ESCAPES_FIRST+0x2b,
+ /* 0xad */ ESCAPES_FIRST+0x2c, /* 0xae */ ESCAPES_FIRST+0x2d,
+ /* 0xaf */ ESCAPES_FIRST+0x2e, /* 0xb0 */ ESCAPES_FIRST+0x2f,
+ /* 0xb1 */ ESCAPES_FIRST+0x30, /* 0xb2 */ ESCAPES_FIRST+0x31,
+ /* 0xb3 */ ESCAPES_FIRST+0x32, /* 0xb4 */ ESCAPES_FIRST+0x33,
+ /* 0xb5 */ ESCAPES_FIRST+0x34, /* 0xb6 */ ESCAPES_FIRST+0x35,
+ /* 0xb7 */ ESCAPES_FIRST+0x36, /* 0xb8 */ ESCAPES_FIRST+0x37,
+ /* 0xb9 */ ESCAPES_FIRST+0x38, /* 0xba */ ESCAPES_FIRST+0x39,
+ /* 0xbb */ ESCAPES_FIRST+0x3a, /* 0xbc */ ESCAPES_FIRST+0x3b,
+ /* 0xbd */ ESCAPES_FIRST+0x3c, /* 0xbe */ ESCAPES_FIRST+0x3d,
+ /* 0xbf */ ESCAPES_FIRST+0x3e, /* 0xc0 */ ESCAPES_FIRST+0x3f,
+ /* 0xc1 A */ -ESC_A, /* 0xc2 B */ -ESC_B,
+ /* 0xc3 C */ -ESC_C, /* 0xc4 D */ -ESC_D,
+ /* 0xc5 E */ -ESC_E, /* 0xc6 F */ 0,
+ /* 0xc7 G */ -ESC_G, /* 0xc8 H */ -ESC_H,
+ /* 0xc9 I */ 0, /* 0xca */ ESCAPES_FIRST+0x49,
+ /* 0xcb */ ESCAPES_FIRST+0x4a, /* 0xcc */ ESCAPES_FIRST+0x4b,
+ /* 0xcd */ ESCAPES_FIRST+0x4c, /* 0xce */ ESCAPES_FIRST+0x4d,
+ /* 0xcf */ ESCAPES_FIRST+0x4e, /* 0xd0 */ ESCAPES_FIRST+0x4f,
+ /* 0xd1 J */ 0, /* 0xd2 K */ -ESC_K,
+ /* 0xd3 L */ 0, /* 0xd4 M */ 0,
+ /* 0xd5 N */ -ESC_N, /* 0xd6 O */ 0,
+ /* 0xd7 P */ -ESC_P, /* 0xd8 Q */ -ESC_Q,
+ /* 0xd9 R */ -ESC_R, /* 0xda */ ESCAPES_FIRST+0x59,
+ /* 0xdb */ ESCAPES_FIRST+0x5a, /* 0xdc */ ESCAPES_FIRST+0x5b,
+ /* 0xdd */ ESCAPES_FIRST+0x5c, /* 0xde */ ESCAPES_FIRST+0x5d,
+ /* 0xdf */ ESCAPES_FIRST+0x5e, /* 0xe0 */ ESCAPES_FIRST+0x5f,
+ /* 0xe1 */ ESCAPES_FIRST+0x60, /* 0xe2 S */ -ESC_S,
+ /* 0xe3 T */ 0, /* 0xe4 U */ 0,
+ /* 0xe5 V */ -ESC_V, /* 0xe6 W */ -ESC_W,
+ /* 0xe7 X */ -ESC_X, /* 0xe8 Y */ 0,
+ /* 0xe9 Z */ -ESC_Z, /* 0xea */ ESCAPES_FIRST+0x69,
+ /* 0xeb */ ESCAPES_FIRST+0x6a, /* 0xec */ ESCAPES_FIRST+0x6b,
+ /* 0xed */ ESCAPES_FIRST+0x6c, /* 0xee */ ESCAPES_FIRST+0x6d,
+ /* 0xef */ ESCAPES_FIRST+0x6e, /* 0xf0 0 */ 0,
+ /* 0xf1 1 */ 0, /* 0xf2 2 */ 0,
+ /* 0xf3 3 */ 0, /* 0xf4 4 */ 0,
+ /* 0xf5 5 */ 0, /* 0xf6 6 */ 0,
+ /* 0xf7 7 */ 0, /* 0xf8 8 */ 0,
+ /* 0xf9 9 */ 0,
};
/* We also need a table of characters that may follow \c in an EBCDIC
environment for characters 0-31. */
-static unsigned char ebcdic_escape_c[] = "@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_";
+static unsigned char ebcdic_escape_c[] = {
+ CHAR_COMMERCIAL_AT, CHAR_A, CHAR_B, CHAR_C, CHAR_D, CHAR_E, CHAR_F, CHAR_G,
+ CHAR_H, CHAR_I, CHAR_J, CHAR_K, CHAR_L, CHAR_M, CHAR_N, CHAR_O, CHAR_P,
+ CHAR_Q, CHAR_R, CHAR_S, CHAR_T, CHAR_U, CHAR_V, CHAR_W, CHAR_X, CHAR_Y,
+ CHAR_Z, CHAR_LEFT_SQUARE_BRACKET, CHAR_BACKSLASH, CHAR_RIGHT_SQUARE_BRACKET,
+ CHAR_CIRCUMFLEX_ACCENT, CHAR_UNDERSCORE
+};
#endif /* EBCDIC */
@@ -1037,12 +1070,12 @@ for (;;)
fprintf(stderr, "META (*scan_substring:");
break;
- case META_SCS_NAME:
- fprintf(stderr, "META_SCS_NAME length=%d relative_offset=%d", *pptr++, (int)meta_arg);
+ case META_CAPTURE_NAME:
+ fprintf(stderr, "META_CAPTURE_NAME length=%d relative_offset=%d", *pptr++, (int)meta_arg);
break;
- case META_SCS_NUMBER:
- fprintf(stderr, "META_SCS_NUMBER %d relative_offset=%d", *pptr++, (int)meta_arg);
+ case META_CAPTURE_NUMBER:
+ fprintf(stderr, "META_CAPTURE_NUMBER %d relative_offset=%d", *pptr++, (int)meta_arg);
break;
case META_MARK:
@@ -1533,6 +1566,13 @@ else if ((i = escapes[c - ESCAPES_FIRST]) != 0)
goto COME_FROM_NU;
}
#endif
+
+ /* Improve error offset. */
+ ptr = p + 2;
+ while (ptr < ptrend && XDIGIT(*ptr) != 0xff) ptr++;
+ while (ptr < ptrend && (*ptr == CHAR_SPACE || *ptr == CHAR_HT)) ptr++;
+ if (ptr < ptrend && *ptr == CHAR_RIGHT_CURLY_BRACKET) ptr++;
+
*errorcodeptr = ERR93;
}
@@ -1541,6 +1581,7 @@ else if ((i = escapes[c - ESCAPES_FIRST]) != 0)
else if (isclass || cb == NULL)
{
+ ptr++; /* Skip over the opening brace */
*errorcodeptr = ERR37;
}
@@ -1551,7 +1592,10 @@ else if ((i = escapes[c - ESCAPES_FIRST]) != 0)
{
if (!read_repeat_counts(&p, ptrend, NULL, NULL, errorcodeptr) &&
*errorcodeptr == 0)
+ {
+ ptr++; /* Skip over the opening brace */
*errorcodeptr = ERR37;
+ }
}
}
}
@@ -1572,11 +1616,11 @@ else
if (cb == NULL)
{
- if (c < CHAR_0 ||
- (c > CHAR_9 && (c != CHAR_c && c != CHAR_o && c != CHAR_x && c != CHAR_g)))
+ if (!(c >= CHAR_0 && c <= CHAR_9) && c != CHAR_c && c != CHAR_o &&
+ c != CHAR_x && c != CHAR_g)
{
*errorcodeptr = ERR3;
- return 0;
+ goto EXIT;
}
alt_bsux = FALSE; /* Do not modify \x handling */
}
@@ -1601,7 +1645,9 @@ else
because otherwise \u{ 12} (for example) would be treated as u{12}. */
case CHAR_u:
- if (!alt_bsux) *errorcodeptr = ERR37; else
+ if (!alt_bsux)
+ *errorcodeptr = ERR37;
+ else
{
uint32_t xc;
@@ -1727,8 +1773,8 @@ else
if (p >= ptrend || *p != CHAR_GREATER_THAN_SIGN)
{
- /* not advancing ptr; report error at the \g character */
- *errorcodeptr = ERR57;
+ ptr = p;
+ *errorcodeptr = ERR119; /* Missing terminator for number */
break;
}
@@ -1764,8 +1810,8 @@ else
if (p >= ptrend || *p != CHAR_RIGHT_CURLY_BRACKET)
{
- /* not advancing ptr; report error at the \g character */
- *errorcodeptr = ERR57;
+ ptr = p;
+ *errorcodeptr = ERR119; /* Missing terminator for number */
break;
}
ptr = p + 1;
@@ -1890,7 +1936,7 @@ else
if (c >= CHAR_8) break;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
/* \0 always starts an octal number, but we may drop through to here with a
larger first octal digit. The original code used just to take the least
@@ -1922,12 +1968,12 @@ else
with optional spaces or tabs after { and before }. */
case CHAR_o:
- if (ptr >= ptrend || *ptr++ != CHAR_LEFT_CURLY_BRACKET)
+ if (ptr >= ptrend || *ptr != CHAR_LEFT_CURLY_BRACKET)
{
- ptr--;
*errorcodeptr = ERR55;
break;
}
+ ptr++;
while (ptr < ptrend && (*ptr == CHAR_SPACE || *ptr == CHAR_HT)) ptr++;
if (ptr >= ptrend || *ptr == CHAR_RIGHT_CURLY_BRACKET)
@@ -1962,19 +2008,19 @@ else
while (ptr < ptrend && *ptr >= CHAR_0 && *ptr <= CHAR_7) ptr++;
*errorcodeptr = ERR34;
}
- else if (ptr < ptrend && *ptr++ == CHAR_RIGHT_CURLY_BRACKET)
+ else if (utf && c >= 0xd800 && c <= 0xdfff &&
+ (xoptions & PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES) == 0)
{
- if (utf && c >= 0xd800 && c <= 0xdfff &&
- (xoptions & PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES) == 0)
- {
- ptr--;
- *errorcodeptr = ERR73;
- }
+ *errorcodeptr = ERR73;
+ }
+ else if (ptr < ptrend && *ptr == CHAR_RIGHT_CURLY_BRACKET)
+ {
+ ptr++;
}
else
{
- ptr--;
*errorcodeptr = ERR64;
+ goto ESCAPE_FAILED_FORWARD;
}
break;
@@ -2043,14 +2089,14 @@ else
while (ptr < ptrend && XDIGIT(*ptr) != 0xff) ptr++;
*errorcodeptr = ERR34;
}
- else if (ptr < ptrend && *ptr++ == CHAR_RIGHT_CURLY_BRACKET)
+ else if (utf && c >= 0xd800 && c <= 0xdfff &&
+ (xoptions & PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES) == 0)
{
- if (utf && c >= 0xd800 && c <= 0xdfff &&
- (xoptions & PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES) == 0)
- {
- ptr--;
- *errorcodeptr = ERR73;
- }
+ *errorcodeptr = ERR73;
+ }
+ else if (ptr < ptrend && *ptr == CHAR_RIGHT_CURLY_BRACKET)
+ {
+ ptr++;
}
/* If the sequence of hex digits (followed by optional space) does not
@@ -2060,8 +2106,8 @@ else
else
{
- ptr--;
*errorcodeptr = ERR67;
+ goto ESCAPE_FAILED_FORWARD;
}
} /* End of \x{} processing */
@@ -2113,11 +2159,7 @@ else
For testing the EBCDIC handling of \c in an ASCII environment, recognize
the EBCDIC value of 'c' explicitly. */
-#if defined EBCDIC && 'a' != 0x81
- case 0x83:
-#else
case CHAR_c:
-#endif
if (ptr >= ptrend)
{
*errorcodeptr = ERR2;
@@ -2132,7 +2174,7 @@ else
if (c < 32 || c > 126) /* Excludes all non-printable ASCII */
{
*errorcodeptr = ERR68;
- break;
+ goto ESCAPE_FAILED_FORWARD;
}
c ^= 0x40;
@@ -2143,14 +2185,20 @@ else
#else
if (c == CHAR_QUESTION_MARK)
- c = ('\\' == 188 && '`' == 74)? 0x5f : 0xff;
+ c = (CHAR_BACKSLASH == 188 && CHAR_GRAVE_ACCENT == 74)? 0x5f : 0xff;
else
{
for (i = 0; i < 32; i++)
{
if (c == ebcdic_escape_c[i]) break;
}
- if (i < 32) c = i; else *errorcodeptr = ERR68;
+ if (i < 32)
+ c = i;
+ else
+ {
+ *errorcodeptr = ERR68;
+ goto ESCAPE_FAILED_FORWARD;
+ }
}
#endif /* EBCDIC */
@@ -2162,16 +2210,25 @@ else
default:
*errorcodeptr = ERR3;
- *ptrptr = ptr - 1; /* Point to the character at fault */
- return 0;
+ break;
}
}
/* Set the pointer to the next character before returning. */
+EXIT:
*ptrptr = ptr;
*chptr = c;
return escape;
+
+/* Some errors need to indicate the next character. */
+
+ESCAPE_FAILED_FORWARD:
+ptr++;
+#ifdef SUPPORT_UNICODE
+if (utf) FORWARDCHARTEST(ptr, ptrend);
+#endif
+goto EXIT;
}
@@ -2188,6 +2245,7 @@ after the final code unit of the escape sequence.
Arguments:
ptrptr the pattern position pointer
+ utf true if the input is UTF-encoded
negptr a boolean that is set TRUE for negation else FALSE
ptypeptr an unsigned int that is set to the type value
pdataptr an unsigned int that is set to the detailed property value
@@ -2198,18 +2256,23 @@ Returns: TRUE if the type value was found, or FALSE for an invalid type
*/
static BOOL
-get_ucp(PCRE2_SPTR *ptrptr, BOOL *negptr, uint16_t *ptypeptr,
+get_ucp(PCRE2_SPTR *ptrptr, BOOL utf, BOOL *negptr, uint16_t *ptypeptr,
uint16_t *pdataptr, int *errorcodeptr, compile_block *cb)
{
-PCRE2_UCHAR c;
-PCRE2_SIZE i, bot, top;
+uint32_t c;
+ptrdiff_t i;
+PCRE2_SIZE bot, top;
PCRE2_SPTR ptr = *ptrptr;
PCRE2_UCHAR name[50];
PCRE2_UCHAR *vptr = NULL;
uint16_t ptscript = PT_NOTSCRIPT;
+#ifndef MAYBE_UTF_MULTI
+(void)utf; /* Avoid compiler warning */
+#endif
+
if (ptr >= cb->end_pattern) goto ERROR_RETURN;
-c = *ptr++;
+GETCHARINCTEST(c, ptr);
*negptr = FALSE;
/* \P or \p can be followed by a name in {}, optionally preceded by ^ for
@@ -2230,15 +2293,14 @@ if (c == CHAR_LEFT_CURLY_BRACKET)
REDO:
if (ptr >= cb->end_pattern) goto ERROR_RETURN;
- c = *ptr++;
+ GETCHARINCTEST(c, ptr);
/* Skip ignorable Unicode characters. */
- while (c == CHAR_UNDERSCORE || c == CHAR_MINUS || c == CHAR_SPACE ||
- (c >= CHAR_HT && c <= CHAR_CR))
+ if (c == CHAR_UNDERSCORE || c == CHAR_MINUS || c == CHAR_SPACE ||
+ (c >= CHAR_HT && c <= CHAR_CR))
{
- if (ptr >= cb->end_pattern) goto ERROR_RETURN;
- c = *ptr++;
+ goto REDO;
}
/* The first significant character being circumflex negates the meaning of
@@ -2506,7 +2568,7 @@ return -1;
the name of a subpattern or a (*VERB) or an (*alpha_assertion). The initial
pointer must be to the preceding character. If that character is '*' we are
reading a verb or alpha assertion name. The pointer is updated to point after
-the name, for a VERB or alpha assertion name, or after tha name's terminator
+the name, for a VERB or alpha assertion name, or after the name's terminator
for a subpattern name. Returning both the offset and the name pointer is
redundant information, but some callers use one and some the other, so it is
simplest just to return both. When the name is in braces, spaces and tabs are
@@ -2559,12 +2621,14 @@ by Unicode properties, and underscores, but must not start with a digit. */
if (utf && is_group)
{
uint32_t c, type;
+ PCRE2_SPTR p = ptr;
- GETCHAR(c, ptr);
+ GETCHARINC(c, p); /* Peek at next character */
type = UCD_CHARTYPE(c);
if (type == ucp_Nd)
{
+ ptr = p;
*errorcodeptr = ERR44;
goto FAILED;
}
@@ -2573,10 +2637,9 @@ if (utf && is_group)
{
if (type != ucp_Nd && PRIV(ucp_gentype)[type] != ucp_L &&
c != CHAR_UNDERSCORE) break;
- ptr++;
- FORWARDCHARTEST(ptr, ptrend);
- if (ptr >= ptrend) break;
- GETCHAR(c, ptr);
+ ptr = p; /* Accept character and peek again */
+ if (p >= ptrend) break;
+ GETCHARINC(c, p);
type = UCD_CHARTYPE(c);
}
}
@@ -2592,6 +2655,7 @@ won't be recognized. */
{
if (is_group && IS_DIGIT(*ptr))
{
+ ++ptr;
*errorcodeptr = ERR44;
goto FAILED;
}
@@ -2604,7 +2668,7 @@ won't be recognized. */
/* Check name length */
-if (ptr > *nameptr + MAX_NAME_SIZE)
+if (ptr - *nameptr > MAX_NAME_SIZE)
{
*errorcodeptr = ERR48;
goto FAILED;
@@ -2623,12 +2687,15 @@ if (is_group)
}
if (is_braced)
while (ptr < ptrend && (*ptr == CHAR_SPACE || *ptr == CHAR_HT)) ptr++;
- if (ptr >= ptrend || *ptr != (PCRE2_UCHAR)terminator)
+ if (terminator != 0)
{
- *errorcodeptr = ERR42;
- goto FAILED;
+ if (ptr >= ptrend || *ptr != (PCRE2_UCHAR)terminator)
+ {
+ *errorcodeptr = ERR42;
+ goto FAILED;
+ }
+ ptr++;
}
- ptr++;
}
*ptrptr = ptr;
@@ -2641,6 +2708,128 @@ return FALSE;
+/**************************************************
+* Parse capturing bracket argument list *
+**************************************************/
+
+/* Reads a list of capture references. The references
+can be numbers or names.
+
+Arguments:
+ ptrptr points to the character pointer variable
+ ptrend points to the end of the input string
+ utf true if the input is UTF-encoded
+ parsed_pattern the parsed pattern pointer
+ offset last known offset
+ errcodeptr where to put an error code
+ cb pointer to the compile data block
+
+Returns: updated parsed_pattern pointer on success
+ NULL otherwise
+*/
+
+static uint32_t *
+parse_capture_list(PCRE2_SPTR *ptrptr, PCRE2_SPTR ptrend,
+ BOOL utf, uint32_t *parsed_pattern, PCRE2_SIZE offset,
+ int *errorcodeptr, compile_block *cb)
+{
+PCRE2_SIZE next_offset;
+PCRE2_SPTR ptr = *ptrptr;
+PCRE2_SPTR name;
+PCRE2_UCHAR terminator;
+uint32_t meta, namelen;
+int i;
+
+if (ptr >= ptrend || *ptr != CHAR_LEFT_PARENTHESIS)
+ {
+ *errorcodeptr = ERR118;
+ goto FAILED;
+ }
+
+for (;;)
+ {
+ ptr++;
+ next_offset = (PCRE2_SIZE)(ptr - cb->start_pattern);
+
+ if (ptr >= ptrend)
+ {
+ *errorcodeptr = ERR117;
+ goto FAILED;
+ }
+
+ /* Handle [+-]number cases */
+ if (read_number(&ptr, ptrend, cb->bracount, MAX_GROUP_NUMBER, ERR61,
+ &i, errorcodeptr))
+ {
+ PCRE2_ASSERT(i >= 0);
+ if (i <= 0)
+ {
+ *errorcodeptr = ERR15;
+ goto FAILED;
+ }
+ meta = META_CAPTURE_NUMBER;
+ namelen = (uint32_t)i;
+ }
+ else if (*errorcodeptr != 0) goto FAILED; /* Number too big */
+ else
+ {
+ /* Handle 'name' or cases. */
+ if (*ptr == CHAR_LESS_THAN_SIGN)
+ terminator = CHAR_GREATER_THAN_SIGN;
+ else if (*ptr == CHAR_APOSTROPHE)
+ terminator = CHAR_APOSTROPHE;
+ else
+ {
+ *errorcodeptr = ERR117;
+ goto FAILED;
+ }
+
+ if (!read_name(&ptr, ptrend, utf, terminator, &next_offset,
+ &name, &namelen, errorcodeptr, cb)) goto FAILED;
+
+ meta = META_CAPTURE_NAME;
+ }
+
+ PCRE2_ASSERT(next_offset > 0);
+ if (offset == 0 || (next_offset - offset) >= 0x10000)
+ {
+ *parsed_pattern++ = META_OFFSET;
+ PUTOFFSET(next_offset, parsed_pattern);
+ offset = next_offset;
+ }
+
+ /* The offset is encoded as a relative offset, because for some
+ inputs such as ",2" in (1,2,3), we only have space for two uint32_t
+ values, and an opcode and absolute offset may require three uint32_t
+ values. */
+ *parsed_pattern++ = meta | (uint32_t)(next_offset - offset);
+ *parsed_pattern++ = namelen;
+ offset = next_offset;
+
+ if (ptr >= ptrend) goto UNCLOSED_PARENTHESIS;
+
+ if (*ptr == CHAR_RIGHT_PARENTHESIS) break;
+
+ if (*ptr != CHAR_COMMA)
+ {
+ *errorcodeptr = ERR24;
+ goto FAILED;
+ }
+ }
+
+*ptrptr = ptr + 1;
+return parsed_pattern;
+
+UNCLOSED_PARENTHESIS:
+*errorcodeptr = ERR14;
+
+FAILED:
+*ptrptr = ptr;
+return NULL;
+}
+
+
+
/*************************************************
* Manage callouts at start of cycle *
*************************************************/
@@ -2719,21 +2908,21 @@ switch(escape)
{
case ESC_D:
prop = ESC_P;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case ESC_d:
ascii_option = PCRE2_EXTRA_ASCII_BSD;
break;
case ESC_S:
prop = ESC_P;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case ESC_s:
ascii_option = PCRE2_EXTRA_ASCII_BSS;
break;
case ESC_W:
prop = ESC_P;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case ESC_w:
ascii_option = PCRE2_EXTRA_ASCII_BSW;
break;
@@ -2942,6 +3131,7 @@ uint32_t add_after_mark = 0;
uint16_t nest_depth = 0;
int16_t class_depth_m1 = -1; /* The m1 means minus 1. */
int16_t class_maxdepth_m1 = -1;
+uint16_t hash;
int after_manual_callout = 0;
int expect_cond_assert = 0;
int errorcode = 0;
@@ -2951,7 +3141,7 @@ BOOL inescq = FALSE;
BOOL inverbname = FALSE;
BOOL utf = (options & PCRE2_UTF) != 0;
BOOL auto_callout = (options & PCRE2_AUTO_CALLOUT) != 0;
-BOOL isdupname;
+BOOL is_dupname;
BOOL negate_class;
BOOL okquantifier = FALSE;
PCRE2_SPTR thisptr;
@@ -2996,12 +3186,15 @@ if ((options & PCRE2_LITERAL) != 0)
{
while (ptr < ptrend)
{
+ /* LCOV_EXCL_START */
if (parsed_pattern >= parsed_pattern_end)
{
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR63; /* Internal error (parsed pattern overflow) */
goto FAILED;
}
+ /* LCOV_EXCL_STOP */
+
thisptr = ptr;
GETCHARINCTEST(c, ptr);
if (auto_callout)
@@ -3067,6 +3260,7 @@ while (ptr < ptrend)
ptr_check = ptr;
#endif
+ /* LCOV_EXCL_START */
if (parsed_pattern >= parsed_pattern_end)
{
/* Weak pre-write check; only ensures parsed_pattern[0] is writeable
@@ -3075,6 +3269,7 @@ while (ptr < ptrend)
errorcode = ERR63; /* Internal error (parsed pattern overflow) */
goto FAILED;
}
+ /* LCOV_EXCL_STOP */
/* If the last time round this loop something was added, parsed_pattern will
no longer be equal to this_parsed_item. Remember where the previous item
@@ -3104,12 +3299,6 @@ while (ptr < ptrend)
}
else
{
- if (expect_cond_assert > 0) /* A literal is not allowed if we are */
- { /* expecting a conditional assertion, */
- ptr--; /* but an empty \Q\E sequence is OK. */
- errorcode = ERR28;
- goto FAILED;
- }
if (inverbname)
{ /* Don't use PARSED_LITERAL() because it */
#if PCRE2_CODE_UNIT_WIDTH == 32 /* sets okquantifier. */
@@ -3238,6 +3427,15 @@ while (ptr < ptrend)
{
if (*ptr == CHAR_Q || *ptr == CHAR_E)
{
+ /* A literal inside a \Q...\E is not allowed if we are expecting a
+ conditional assertion, but an empty \Q\E sequence is OK. */
+ if (expect_cond_assert > 0 && *ptr == CHAR_Q &&
+ !(ptrend - ptr >= 3 && ptr[1] == CHAR_BACKSLASH && ptr[2] == CHAR_E))
+ {
+ ptr--;
+ errorcode = ERR28;
+ goto FAILED;
+ }
inescq = *ptr == CHAR_Q;
ptr++;
continue;
@@ -3347,9 +3545,9 @@ while (ptr < ptrend)
if (!ok)
{
- ptr--; /* Adjust error offset */
errorcode = ERR28;
- goto FAILED;
+ if (expect_cond_assert == 2) goto FAILED;
+ goto FAILED_BACK;
}
}
@@ -3425,7 +3623,7 @@ while (ptr < ptrend)
else if (escape < 0)
{
- offset = (PCRE2_SIZE)(ptr - cb->start_pattern - 1);
+ offset = (PCRE2_SIZE)(ptr - cb->start_pattern);
escape = -escape - 1;
*parsed_pattern++ = META_BACKREF | (uint32_t)escape;
if (escape < 10)
@@ -3523,7 +3721,7 @@ while (ptr < ptrend)
{
BOOL negated;
uint16_t ptype = 0, pdata = 0;
- if (!get_ucp(&ptr, &negated, &ptype, &pdata, &errorcode, cb))
+ if (!get_ucp(&ptr, utf, &negated, &ptype, &pdata, &errorcode, cb))
goto ESCAPE_FAILED;
if (negated) escape = (escape == ESC_P)? ESC_p : ESC_P;
*parsed_pattern++ = META_ESCAPE + escape;
@@ -3565,10 +3763,11 @@ while (ptr < ptrend)
{
if (p >= ptrend || *p != terminator)
{
- errorcode = ERR57;
+ ptr = p;
+ errorcode = ERR119; /* Missing terminator for number */
goto ESCAPE_FAILED;
}
- ptr = p;
+ ptr = p + 1;
goto SET_RECURSION;
}
if (errorcode != 0) goto ESCAPE_FAILED;
@@ -3649,7 +3848,7 @@ while (ptr < ptrend)
if (!prev_okquantifier)
{
errorcode = ERR9;
- goto FAILED_BACK; // TODO https://github.com/PCRE2Project/pcre2/issues/549
+ goto FAILED;
}
/* Most (*VERB)s are not allowed to be quantified, but an ungreedy
@@ -3738,6 +3937,7 @@ while (ptr < ptrend)
check_posix_syntax(ptr, ptrend, &tempptr))
{
errorcode = (*ptr-- == CHAR_COLON)? ERR12 : ERR13;
+ ptr = tempptr + 2;
goto FAILED;
}
@@ -3965,8 +4165,9 @@ while (ptr < ptrend)
/* Validate nesting depth */
if (class_depth_m1 >= ECLASS_NEST_LIMIT - 1)
{
- errorcode = ERR107;
- goto FAILED; /* Classes too deeply nested */
+ ptr--; /* Point rightwards at the paren, same as ERR19. */
+ errorcode = ERR107; /* Classes too deeply nested */
+ goto FAILED;
}
/* Process the character class start. If the first character is '^', set
@@ -4081,7 +4282,8 @@ while (ptr < ptrend)
if (c == CHAR_RIGHT_SQUARE_BRACKET && class_depth_m1 != 0)
{
errorcode = ERR14;
- goto FAILED_BACK;
+ ptr--; /* Correct the offset */
+ goto FAILED;
}
if (c == CHAR_RIGHT_PARENTHESIS && class_depth_m1 < 1)
{
@@ -4301,7 +4503,6 @@ while (ptr < ptrend)
case ESC_R:
case ESC_X:
errorcode = ERR7;
- ptr--; // TODO https://github.com/PCRE2Project/pcre2/issues/549
goto FAILED;
case ESC_N: /* Not permitted by Perl either */
@@ -4336,7 +4537,7 @@ while (ptr < ptrend)
{
BOOL negated;
uint16_t ptype = 0, pdata = 0;
- if (!get_ucp(&ptr, &negated, &ptype, &pdata, &errorcode, cb))
+ if (!get_ucp(&ptr, utf, &negated, &ptype, &pdata, &errorcode, cb))
goto FAILED;
/* In caseless matching, particular characteristics Lu, Ll, and Lt
@@ -4362,9 +4563,11 @@ while (ptr < ptrend)
/* All others are not allowed in a class */
+ /* LCOV_EXCL_START */
default:
PCRE2_DEBUG_UNREACHABLE();
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
+ /* LCOV_EXCL_STOP */
case ESC_A:
case ESC_Z:
@@ -4373,7 +4576,6 @@ while (ptr < ptrend)
case ESC_K:
case ESC_C:
errorcode = ERR7;
- ptr--; // TODO https://github.com/PCRE2Project/pcre2/issues/549
goto FAILED;
}
@@ -4464,7 +4666,7 @@ while (ptr < ptrend)
else if (parsed_pattern[-2] > c) /* Check range is in order */
{
errorcode = ERR8;
- goto FAILED_BACK; // TODO https://github.com/PCRE2Project/pcre2/issues/549
+ goto FAILED;
}
else
{
@@ -4560,10 +4762,11 @@ while (ptr < ptrend)
vn = alasnames;
if (!read_name(&ptr, ptrend, utf, 0, &offset, &name, &namelen,
&errorcode, cb)) goto FAILED;
- if (ptr >= ptrend || *ptr != CHAR_COLON)
+ if (ptr >= ptrend) goto UNCLOSED_PARENTHESIS;
+ if (*ptr != CHAR_COLON)
{
errorcode = ERR95; /* Malformed */
- goto FAILED;
+ goto FAILED_FORWARD;
}
/* Scan the table of alpha assertion names */
@@ -4598,10 +4801,12 @@ while (ptr < ptrend)
switch(meta)
{
+ /* LCOV_EXCL_START */
default:
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR89; /* Unknown code; should never occur because */
goto FAILED; /* the meta values come from a table above. */
+ /* LCOV_EXCL_STOP */
case META_ATOMIC:
goto ATOMIC_GROUP;
@@ -4616,87 +4821,12 @@ while (ptr < ptrend)
goto NEGATIVE_LOOK_AHEAD;
case META_SCS:
- if (++ptr >= ptrend) goto UNCLOSED_PARENTHESIS;
-
- if (*ptr != CHAR_LEFT_PARENTHESIS)
- {
- errorcode = ERR15;
- goto FAILED;
- }
-
ptr++;
*parsed_pattern++ = META_SCS;
- /* Temporary variable, zero in the first iteration. */
- offset = 0;
-
- for (;;)
- {
- PCRE2_SIZE next_offset = (PCRE2_SIZE)(ptr - cb->start_pattern);
-
- /* Handle (scan_substring:([+-]number)... */
- if (read_number(&ptr, ptrend, cb->bracount, MAX_GROUP_NUMBER, ERR61,
- &i, &errorcode))
- {
- PCRE2_ASSERT(i >= 0);
- if (i <= 0)
- {
- errorcode = ERR15;
- goto FAILED;
- }
- meta = META_SCS_NUMBER;
- namelen = (uint32_t)i;
- }
- else if (errorcode != 0) goto FAILED; /* Number too big */
- else
- {
- if (ptr >= ptrend) goto UNCLOSED_PARENTHESIS;
- /* Handle (*scan_substring:('name') or (*scan_substring:() */
- if (*ptr == CHAR_LESS_THAN_SIGN)
- terminator = CHAR_GREATER_THAN_SIGN;
- else if (*ptr == CHAR_APOSTROPHE)
- terminator = CHAR_APOSTROPHE;
- else
- {
- errorcode = ERR15;
- goto FAILED;
- }
-
- if (!read_name(&ptr, ptrend, utf, terminator, &next_offset,
- &name, &namelen, &errorcode, cb)) goto FAILED;
-
- meta = META_SCS_NAME;
- }
-
- PCRE2_ASSERT(next_offset > 0);
- if (offset == 0 || (next_offset - offset) >= 0x10000)
- {
- *parsed_pattern++ = META_OFFSET;
- PUTOFFSET(next_offset, parsed_pattern);
- offset = next_offset;
- }
-
- /* The offset is encoded as a relative offset, because for some
- inputs such as ",2" in (*scs:(1,2,3)...), we only have space for
- two uint32_t values, and an opcode and absolute offset may require
- three uint32_t values. */
- *parsed_pattern++ = meta | (uint32_t)(next_offset - offset);
- *parsed_pattern++ = namelen;
- offset = next_offset;
-
- if (ptr >= ptrend) goto UNCLOSED_PARENTHESIS;
-
- if (*ptr == CHAR_RIGHT_PARENTHESIS) break;
-
- if (*ptr != CHAR_COMMA)
- {
- errorcode = ERR24;
- goto FAILED;
- }
-
- ptr++;
- }
- ptr++;
+ parsed_pattern = parse_capture_list(&ptr, ptrend, utf, parsed_pattern,
+ 0, &errorcode, cb);
+ if (parsed_pattern == NULL) goto FAILED;
goto POST_ASSERTION;
case META_LOOKBEHIND:
@@ -4923,7 +5053,6 @@ while (ptr < ptrend)
if (!hyphenok)
{
errorcode = ERR94;
- ptr--; /* Correct the offset */
goto FAILED;
}
optset = &unset;
@@ -4997,7 +5126,6 @@ while (ptr < ptrend)
default:
errorcode = ERR11;
- ptr--; /* Correct the offset */
goto FAILED;
}
}
@@ -5066,7 +5194,7 @@ while (ptr < ptrend)
if (*ptr != CHAR_EQUALS_SIGN)
{
errorcode = ERR41;
- goto FAILED;
+ goto FAILED_FORWARD;
}
if (!read_name(&ptr, ptrend, utf, CHAR_RIGHT_PARENTHESIS, &offset, &name,
&namelen, &errorcode, cb)) goto FAILED;
@@ -5082,23 +5210,30 @@ while (ptr < ptrend)
case CHAR_R:
i = 0; /* (?R) == (?R0) */
ptr++;
- if (ptr >= ptrend || *ptr != CHAR_RIGHT_PARENTHESIS)
+ if (ptr >= ptrend || (*ptr != CHAR_RIGHT_PARENTHESIS && *ptr != CHAR_LEFT_PARENTHESIS))
{
errorcode = ERR58;
goto FAILED;
}
+ terminator = CHAR_NUL;
goto SET_RECURSION;
/* An item starting (?- followed by a digit comes here via the "default"
case because (?- followed by a non-digit is an options setting. */
case CHAR_PLUS:
- if (ptrend - ptr < 2 || !IS_DIGIT(ptr[1]))
+ if (ptr + 1 >= ptrend)
+ {
+ ++ptr;
+ goto UNCLOSED_PARENTHESIS;
+ }
+ if (!IS_DIGIT(ptr[1]))
{
errorcode = ERR29; /* Missing number */
- goto FAILED;
+ ++ptr;
+ goto FAILED_FORWARD;
}
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case CHAR_0: case CHAR_1: case CHAR_2: case CHAR_3: case CHAR_4:
case CHAR_5: case CHAR_6: case CHAR_7: case CHAR_8: case CHAR_9:
@@ -5108,28 +5243,43 @@ while (ptr < ptrend)
MAX_GROUP_NUMBER, ERR61,
&i, &errorcode)) goto FAILED;
PCRE2_ASSERT(i >= 0); /* NB (?0) is permitted, represented by i=0 */
- if (ptr >= ptrend || *ptr != CHAR_RIGHT_PARENTHESIS)
- goto UNCLOSED_PARENTHESIS;
+ terminator = CHAR_NUL;
SET_RECURSION:
*parsed_pattern++ = META_RECURSE | (uint32_t)i;
offset = (PCRE2_SIZE)(ptr - cb->start_pattern);
- ptr++;
- PUTOFFSET(offset, parsed_pattern);
- okquantifier = TRUE;
- break; /* End of recursive call by number handling */
+ /* End of recursive call by number handling */
+ goto READ_RECURSION_ARGUMENTS;
/* ---- Recursion/subroutine calls by name ---- */
case CHAR_AMPERSAND:
RECURSE_BY_NAME:
- if (!read_name(&ptr, ptrend, utf, CHAR_RIGHT_PARENTHESIS, &offset, &name,
+ if (!read_name(&ptr, ptrend, utf, 0, &offset, &name,
&namelen, &errorcode, cb)) goto FAILED;
*parsed_pattern++ = META_RECURSE_BYNAME;
*parsed_pattern++ = namelen;
+ terminator = CHAR_NUL;
+
+ READ_RECURSION_ARGUMENTS:
PUTOFFSET(offset, parsed_pattern);
okquantifier = TRUE;
+
+ /* Arguments are not supported for \g construct. */
+ if (terminator != CHAR_NUL) break;
+
+ if (ptr < ptrend && *ptr == CHAR_LEFT_PARENTHESIS)
+ {
+ parsed_pattern = parse_capture_list(&ptr, ptrend, utf, parsed_pattern,
+ offset, &errorcode, cb);
+ if (parsed_pattern == NULL) goto FAILED;
+ }
+
+ if (ptr >= ptrend || *ptr != CHAR_RIGHT_PARENTHESIS)
+ goto UNCLOSED_PARENTHESIS;
+
+ ptr++;
break;
/* ---- Callout with numerical or string argument ---- */
@@ -5137,6 +5287,7 @@ while (ptr < ptrend)
case CHAR_C:
if ((xoptions & PCRE2_EXTRA_NEVER_CALLOUT) != 0)
{
+ ptr++;
errorcode = ERR103;
goto FAILED;
}
@@ -5189,7 +5340,7 @@ while (ptr < ptrend)
if (delimiter == 0)
{
errorcode = ERR82;
- goto FAILED;
+ goto FAILED_FORWARD;
}
*parsed_pattern = META_CALLOUT_STRING;
@@ -5333,20 +5484,31 @@ while (ptr < ptrend)
references its argument twice. */
if (*ptr != CHAR_EQUALS_SIGN || (ptr++, !IS_DIGIT(*ptr)))
- goto BAD_VERSION_CONDITION;
+ {
+ errorcode = ERR79;
+ if (!ge) goto FAILED_FORWARD;
+ goto FAILED;
+ }
if (!read_number(&ptr, ptrend, -1, 1000, ERR79, &major, &errorcode))
goto FAILED;
- if (ptr >= ptrend) goto BAD_VERSION_CONDITION;
- if (*ptr == CHAR_DOT)
+ if (ptr < ptrend && *ptr == CHAR_DOT)
{
- if (++ptr >= ptrend || !IS_DIGIT(*ptr)) goto BAD_VERSION_CONDITION;
- minor = (*ptr++ - CHAR_0) * 10;
- if (ptr >= ptrend) goto BAD_VERSION_CONDITION;
- if (IS_DIGIT(*ptr)) minor += *ptr++ - CHAR_0;
- if (ptr >= ptrend || *ptr != CHAR_RIGHT_PARENTHESIS)
- goto BAD_VERSION_CONDITION;
+ if (++ptr >= ptrend || !IS_DIGIT(*ptr))
+ {
+ errorcode = ERR79;
+ if (ptr < ptrend) goto FAILED_FORWARD;
+ goto FAILED;
+ }
+ if (!read_number(&ptr, ptrend, -1, 1000, ERR79, &minor, &errorcode))
+ goto FAILED;
+ }
+ if (ptr >= ptrend || *ptr != CHAR_RIGHT_PARENTHESIS)
+ {
+ errorcode = ERR79;
+ if (ptr < ptrend) goto FAILED_FORWARD;
+ goto FAILED;
}
*parsed_pattern++ = META_COND_VERSION;
@@ -5380,6 +5542,7 @@ while (ptr < ptrend)
terminator = CHAR_RIGHT_PARENTHESIS;
ptr--; /* Point to char before name */
}
+
if (!read_name(&ptr, ptrend, utf, terminator, &offset, &name, &namelen,
&errorcode, cb)) goto FAILED;
@@ -5559,21 +5722,37 @@ while (ptr < ptrend)
scanning in case this is a duplicate with the same number. For
non-duplicate names, give an error if the number is duplicated. */
- isdupname = FALSE;
+ is_dupname = FALSE;
+ hash = PRIV(compile_get_hash_from_name)(name, namelen);
ng = cb->named_groups;
for (i = 0; i < cb->names_found; i++, ng++)
{
- if (namelen == ng->length &&
+ if (namelen == ng->length && hash == NAMED_GROUP_GET_HASH(ng) &&
PRIV(strncmp)(name, ng->name, (PCRE2_SIZE)namelen) == 0)
{
+ /* When a bracket is referenced by the same name multiple
+ times, is not considered as a duplicate and ignored. */
if (ng->number == cb->bracount) break;
if ((options & PCRE2_DUPNAMES) == 0)
{
errorcode = ERR43;
goto FAILED;
}
- isdupname = ng->isdup = TRUE; /* Mark as a duplicate */
+
+ ng->hash_dup |= NAMED_GROUP_IS_DUPNAME;
+ is_dupname = TRUE; /* Mark as a duplicate */
cb->dupnames = TRUE; /* Duplicate names exist */
+
+ /* The entry represents a duplicate. */
+ name = ng->name;
+ namelen = 0;
+
+ /* Even duplicated names may refer to the same
+ capture index. These references are also ignored. */
+ for (; i < cb->names_found; i++, ng++)
+ if (ng->name == name && ng->number == cb->bracount)
+ break;
+ break;
}
else if (ng->number == cb->bracount)
{
@@ -5582,7 +5761,8 @@ while (ptr < ptrend)
}
}
- if (i < cb->names_found) break; /* Ignore duplicate with same number */
+ /* Ignore duplicate with same number. */
+ if (i < cb->names_found) break;
/* Increase the list size if necessary */
@@ -5608,11 +5788,13 @@ while (ptr < ptrend)
}
/* Add this name to the list */
+ if (is_dupname)
+ hash |= NAMED_GROUP_IS_DUPNAME;
cb->named_groups[cb->names_found].name = name;
cb->named_groups[cb->names_found].length = (uint16_t)namelen;
cb->named_groups[cb->names_found].number = cb->bracount;
- cb->named_groups[cb->names_found].isdup = (uint16_t)isdupname;
+ cb->named_groups[cb->names_found].hash_dup = hash;
cb->names_found++;
break;
@@ -5678,7 +5860,7 @@ while (ptr < ptrend)
if (nest_depth == 0) /* Unmatched closing parenthesis */
{
errorcode = ERR22;
- goto FAILED_BACK; // TODO https://github.com/PCRE2Project/pcre2/issues/549
+ goto FAILED;
}
nest_depth--;
*parsed_pattern++ = META_KET;
@@ -5723,12 +5905,14 @@ else if ((xoptions & PCRE2_EXTRA_MATCH_WORD) != 0)
/* Terminate the parsed pattern, then return success if all groups are closed.
Otherwise we have unclosed parentheses. */
+/* LCOV_EXCL_START */
if (parsed_pattern >= parsed_pattern_end)
{
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR63; /* Internal error (parsed pattern overflow) */
goto FAILED;
}
+/* LCOV_EXCL_STOP */
*parsed_pattern = META_END;
if (nest_depth == 0) return 0;
@@ -5746,12 +5930,18 @@ return errorcode;
FAILED_BACK:
ptr--;
+#ifdef SUPPORT_UNICODE
+if (utf) BACKCHAR(ptr);
+#endif
goto FAILED;
-/* This failure happens several times. */
+/* Some errors need to indicate the next character. */
-BAD_VERSION_CONDITION:
-errorcode = ERR79;
+FAILED_FORWARD:
+ptr++;
+#ifdef SUPPORT_UNICODE
+if (utf) FORWARDCHARTEST(ptr, ptrend);
+#endif
goto FAILED;
}
@@ -5795,7 +5985,7 @@ for (;;)
case OP_UCP_WORD_BOUNDARY:
case OP_NOT_UCP_WORD_BOUNDARY:
if (!skipassert) return code;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_CALLOUT:
case OP_CREF:
@@ -5836,78 +6026,9 @@ for (;;)
}
}
+/* LCOV_EXCL_START */
PCRE2_DEBUG_UNREACHABLE(); /* Control should never reach here */
-}
-
-
-
-/*************************************************
-* Find details of duplicate group names *
-*************************************************/
-
-/* This is called from compile_branch() when it needs to know the index and
-count of duplicates in the names table when processing named backreferences,
-either directly, or as conditions.
-
-Arguments:
- name points to the name
- length the length of the name
- indexptr where to put the index
- countptr where to put the count of duplicates
- errorcodeptr where to put an error code
- cb the compile block
-
-Returns: TRUE if OK, FALSE if not, error code set
-*/
-
-static BOOL
-find_dupname_details(PCRE2_SPTR name, uint32_t length, int *indexptr,
- int *countptr, int *errorcodeptr, compile_block *cb)
-{
-uint32_t i, groupnumber;
-int count;
-PCRE2_UCHAR *slot = cb->name_table;
-
-/* Find the first entry in the table */
-
-for (i = 0; i < cb->names_found; i++)
- {
- if (PRIV(strncmp)(name, slot+IMM2_SIZE, length) == 0 &&
- slot[IMM2_SIZE+length] == 0) break;
- slot += cb->name_entry_size;
- }
-
-/* This should not occur, because this function is called only when we know we
-have duplicate names. Give an internal error. */
-
-if (i >= cb->names_found)
- {
- PCRE2_DEBUG_UNREACHABLE();
- *errorcodeptr = ERR53;
- cb->erroroffset = name - cb->start_pattern;
- return FALSE;
- }
-
-/* Record the index and then see how many duplicates there are, updating the
-backref map and maximum back reference as we do. */
-
-*indexptr = i;
-count = 0;
-
-for (;;)
- {
- count++;
- groupnumber = GET2(slot,0);
- cb->backref_map |= (groupnumber < 32)? (1u << groupnumber) : 1;
- if (groupnumber > cb->top_backref) cb->top_backref = groupnumber;
- if (++i >= cb->names_found) break;
- slot += cb->name_entry_size;
- if (PRIV(strncmp)(name, slot+IMM2_SIZE, length) != 0 ||
- (slot+IMM2_SIZE)[length] != 0) break;
- }
-
-*countptr = count;
-return TRUE;
+/* LCOV_EXCL_STOP */
}
@@ -5965,7 +6086,7 @@ uint32_t meta, meta_arg;
uint32_t firstcuflags, reqcuflags;
uint32_t zeroreqcuflags, zerofirstcuflags;
uint32_t req_caseopt, reqvary, tempreqvary;
-/* Some opcodes, such as META_SCS_NUMBER or META_SCS_NAME,
+/* Some opcodes, such as META_CAPTURE_NUMBER or META_CAPTURE_NAME,
depends on the previous value of offset. */
PCRE2_SIZE offset = 0;
PCRE2_SIZE length_prevgroup = 0;
@@ -6042,16 +6163,21 @@ for (;; pptr++)
if (lengthptr != NULL)
{
+ /* LCOV_EXCL_START */
+ if (code >= cb->start_workspace + cb->workspace_size)
+ {
+ PCRE2_DEBUG_UNREACHABLE();
+ *errorcodeptr = ERR52; /* Over-ran workspace - internal error */
+ cb->erroroffset = 0;
+ return 0;
+ }
+ /* LCOV_EXCL_STOP */
+
if (code > cb->start_workspace + cb->workspace_size -
WORK_SIZE_SAFETY_MARGIN) /* Check for overrun */
{
- if (code >= cb->start_workspace + cb->workspace_size)
- {
- PCRE2_DEBUG_UNREACHABLE();
- *errorcodeptr = ERR52; /* Over-ran workspace - internal error */
- }
- else
- *errorcodeptr = ERR86;
+ *errorcodeptr = ERR86; /* Pattern too complicated */
+ cb->erroroffset = 0;
return 0;
}
@@ -6072,12 +6198,14 @@ for (;; pptr++)
if (OFLOW_MAX - *lengthptr < (PCRE2_SIZE)(code - orig_code))
{
*errorcodeptr = ERR20; /* Integer overflow */
+ cb->erroroffset = 0;
return 0;
}
*lengthptr += (PCRE2_SIZE)(code - orig_code);
if (*lengthptr > MAX_PATTERN_SIZE)
{
*errorcodeptr = ERR20; /* Pattern is too large */
+ cb->erroroffset = 0;
return 0;
}
code = orig_code;
@@ -6380,7 +6508,7 @@ for (;; pptr++)
case META_PRUNE:
case META_SKIP:
cb->had_pruneorskip = TRUE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case META_COMMIT:
case META_FAIL:
*code++ = verbops[(meta - META_MARK) >> 16];
@@ -6405,7 +6533,7 @@ for (;; pptr++)
case META_PRUNE_ARG:
case META_SKIP_ARG:
cb->had_pruneorskip = TRUE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case META_MARK:
case META_COMMIT_ARG:
VERB_ARG:
@@ -6450,8 +6578,63 @@ for (;; pptr++)
req_caseopt = ((options & PCRE2_CASELESS) != 0)? REQ_CASELESS : 0;
break;
+ /* ===================================================================*/
+ /* Handle scan substring. Scan substring assertion starts with META_SCS,
+ which recursively calls compile_branch. The first opcode processed by
+ this recursive call is always META_OFFSET. */
+
case META_OFFSET:
- GETPLUSOFFSET(offset, pptr);
+ if (lengthptr != NULL)
+ {
+ pptr = PRIV(compile_parse_scan_substr_args)(pptr, errorcodeptr, cb, lengthptr);
+ if (pptr == NULL)
+ return 0;
+ break;
+ }
+
+ while (TRUE)
+ {
+ int count, index;
+ named_group *ng;
+
+ switch (META_CODE(*pptr))
+ {
+ case META_OFFSET:
+ pptr++;
+ SKIPOFFSET(pptr);
+ continue;
+
+ case META_CAPTURE_NAME:
+ ng = cb->named_groups + pptr[1];
+ pptr += 2;
+ count = 0;
+ index = 0;
+
+ if (!PRIV(compile_find_dupname_details)(ng->name, ng->length, &index,
+ &count, errorcodeptr, cb)) return 0;
+
+ code[0] = OP_DNCREF;
+ PUT2(code, 1, index);
+ PUT2(code, 1 + IMM2_SIZE, count);
+ code += 1 + 2 * IMM2_SIZE;
+ continue;
+
+ case META_CAPTURE_NUMBER:
+ pptr += 2;
+ if (pptr[-1] == 0) continue;
+
+ code[0] = OP_CREF;
+ PUT2(code, 1, pptr[-1]);
+ code += 1 + IMM2_SIZE;
+ continue;
+
+ default:
+ break;
+ }
+
+ break;
+ }
+ --pptr;
break;
case META_SCS:
@@ -6470,19 +6653,17 @@ for (;; pptr++)
case META_COND_RNUMBER: /* (?(Rdigits) */
case META_COND_NAME: /* (?(name) or (?'name') or ?() */
case META_COND_RNAME: /* (?(R&name) - test for recursion */
- case META_SCS_NAME: /* Name of scan substring */
bravalue = OP_COND;
+
+ if (lengthptr != NULL)
{
- int count, index;
- unsigned int i;
+ uint32_t i;
PCRE2_SPTR name;
- named_group *ng = cb->named_groups;
+ named_group *ng;
+ uint32_t *start_pptr = pptr;
uint32_t length = *(++pptr);
- if (meta == META_SCS_NAME)
- offset += meta_arg;
- else
- GETPLUSOFFSET(offset, pptr);
+ GETPLUSOFFSET(offset, pptr);
name = cb->start_pattern + offset;
/* In the first pass, the names generated in the pre-pass are available,
@@ -6491,11 +6672,9 @@ for (;; pptr++)
this name is duplicated. If it is not duplicated, we can handle it as a
numerical group. */
- for (i = 0; i < cb->names_found; i++, ng++)
- if (length == ng->length &&
- PRIV(strncmp)(name, ng->name, length) == 0) break;
+ ng = PRIV(compile_find_named_group)(name, length, cb);
- if (i >= cb->names_found)
+ if (ng == NULL)
{
/* If the name was not found we have a bad reference, unless we are
dealing with R, which is treated as a recursion test by
@@ -6528,61 +6707,88 @@ for (;; pptr++)
translated into RREF_ANY (which is 0xffff). */
if (groupnumber == 0) groupnumber = RREF_ANY;
- code[1+LINK_SIZE] = OP_RREF;
- PUT2(code, 2+LINK_SIZE, groupnumber);
+ PCRE2_ASSERT(start_pptr[0] == META_COND_RNUMBER);
+ start_pptr[1] = groupnumber;
skipunits = 1+IMM2_SIZE;
goto GROUP_PROCESS_NOTE_EMPTY;
}
- else if (!ng->isdup)
+
+ /* From here on, we know we have a name (not a number),
+ so treat META_COND_RNUMBER the same as META_COND_NAME. */
+ if (meta == META_COND_RNUMBER) meta = META_COND_NAME;
+
+ if ((ng->hash_dup & NAMED_GROUP_IS_DUPNAME) == 0)
{
- /* Otherwise found a duplicated name */
+ /* Found a non-duplicated name. Since it is a global,
+ it is enough to update it in the pre-processing phase. */
if (ng->number > cb->top_backref) cb->top_backref = ng->number;
- if (meta == META_SCS_NAME)
- {
- code[0] = OP_CREF;
- PUT2(code, 1, ng->number);
- code += 1+IMM2_SIZE;
- break;
- }
+ start_pptr[0] = meta;
+ start_pptr[1] = ng->number;
- code[1+LINK_SIZE] = (meta == META_COND_RNAME)? OP_RREF : OP_CREF;
- PUT2(code, 2+LINK_SIZE, ng->number);
- skipunits = 1+IMM2_SIZE;
- if (meta != META_SCS_NAME) goto GROUP_PROCESS_NOTE_EMPTY;
- cb->assert_depth += 1;
- goto GROUP_PROCESS;
+ skipunits = 1 + IMM2_SIZE;
+ goto GROUP_PROCESS_NOTE_EMPTY;
}
/* We have a duplicated name. In the compile pass we have to search the
main table in order to get the index and count values. */
- count = 0; /* Values for first pass (avoids compiler warning) */
- index = 0;
- if (lengthptr == NULL && !find_dupname_details(name, length, &index,
- &count, errorcodeptr, cb)) return 0;
+ start_pptr[0] = meta | 1;
+ start_pptr[1] = (uint32_t)(ng - cb->named_groups);
- if (meta == META_SCS_NAME)
+ /* A duplicated name was found. Note that if an R name is found
+ (META_COND_RNUMBER), it is a reference test, not a recursion test. */
+ skipunits = 1 + 2 * IMM2_SIZE;
+ }
+ else
+ {
+ /* Otherwise lengthptr equals to NULL,
+ which is the second phase of compilation. */
+ int count, index;
+ named_group *ng;
+
+ /* Generate code using the data
+ collected in the pre-processing phase. */
+
+ if (meta == META_COND_RNUMBER)
{
- code[0] = OP_DNCREF;
- PUT2(code, 1, index);
- PUT2(code, 1+IMM2_SIZE, count);
- code += 1+2*IMM2_SIZE;
- break;
+ code[1+LINK_SIZE] = OP_RREF;
+ PUT2(code, 2 + LINK_SIZE, pptr[1]);
+ skipunits = 1 + IMM2_SIZE;
+ pptr += 1 + SIZEOFFSET;
+ goto GROUP_PROCESS_NOTE_EMPTY;
+ }
+
+ if (meta_arg == 0)
+ {
+ code[1+LINK_SIZE] = (meta == META_COND_RNAME)? OP_RREF : OP_CREF;
+ PUT2(code, 2 + LINK_SIZE, pptr[1]);
+ skipunits = 1 + IMM2_SIZE;
+ pptr += 1 + SIZEOFFSET;
+ goto GROUP_PROCESS_NOTE_EMPTY;
}
+ ng = cb->named_groups + pptr[1];
+ count = 0; /* Values for first pass (avoids compiler warning) */
+ index = 0;
+
+ /* The failed case is an internal error. */
+ if (!PRIV(compile_find_dupname_details)(ng->name, ng->length, &index,
+ &count, errorcodeptr, cb)) return 0;
+
/* A duplicated name was found. Note that if an R name is found
(META_COND_RNUMBER), it is a reference test, not a recursion test. */
- code[1+LINK_SIZE] = (meta == META_COND_RNAME)? OP_DNRREF : OP_DNCREF;
+ code[1 + LINK_SIZE] = (meta == META_COND_RNAME)? OP_DNRREF : OP_DNCREF;
/* Insert appropriate data values. */
- skipunits = 1+2*IMM2_SIZE;
- PUT2(code, 2+LINK_SIZE, index);
- PUT2(code, 2+LINK_SIZE+IMM2_SIZE, count);
+ PUT2(code, 2 + LINK_SIZE, index);
+ PUT2(code, 2 + LINK_SIZE + IMM2_SIZE, count);
+ skipunits = 1 + 2 * IMM2_SIZE;
+ pptr += 1 + SIZEOFFSET;
}
- PCRE2_ASSERT(meta != META_SCS_NAME);
+ PCRE2_ASSERT(meta != META_CAPTURE_NAME);
goto GROUP_PROCESS_NOTE_EMPTY;
/* The DEFINE condition is always false. Its internal groups may never
@@ -6599,12 +6805,8 @@ for (;; pptr++)
/* Conditional test of a group's being set. */
case META_COND_NUMBER:
- case META_SCS_NUMBER:
bravalue = OP_COND;
- if (meta == META_SCS_NUMBER)
- offset += meta_arg;
- else
- GETPLUSOFFSET(offset, pptr);
+ GETPLUSOFFSET(offset, pptr);
groupnumber = *(++pptr);
if (groupnumber > cb->bracount)
@@ -6615,14 +6817,6 @@ for (;; pptr++)
}
if (groupnumber > cb->top_backref) cb->top_backref = groupnumber;
- if (meta == META_SCS_NUMBER)
- {
- code[0] = OP_CREF;
- PUT2(code, 1, groupnumber);
- code += 1+IMM2_SIZE;
- break;
- }
-
/* Point at initial ( for too many branches error */
offset -= 2;
code[1+LINK_SIZE] = OP_CREF;
@@ -6930,8 +7124,7 @@ for (;; pptr++)
{
int count, index;
PCRE2_SPTR name;
- BOOL is_dupname = FALSE;
- named_group *ng = cb->named_groups;
+ named_group *ng;
uint32_t length = *(++pptr);
GETPLUSOFFSET(offset, pptr);
@@ -6942,47 +7135,39 @@ for (;; pptr++)
generated in the pre-pass in order to get a number and whether or not
this name is duplicated. */
- groupnumber = 0;
- for (unsigned int i = 0; i < cb->names_found; i++, ng++)
+ ng = PRIV(compile_find_named_group)(name, length, cb);
+
+ if (ng == NULL)
{
- if (length == ng->length &&
- PRIV(strncmp)(name, ng->name, length) == 0)
- {
- is_dupname = ng->isdup;
- groupnumber = ng->number;
+ /* If the name was not found we have a bad reference. */
+ *errorcodeptr = ERR15;
+ cb->erroroffset = offset;
+ return 0;
+ }
- /* For a recursion, that's all that is needed. We can now go to
- the code that handles numerical recursion, applying it to the first
- group with the given name. */
+ groupnumber = ng->number;
- if (meta == META_RECURSE_BYNAME)
- {
- meta_arg = groupnumber;
- goto HANDLE_NUMERICAL_RECURSION;
- }
-
- /* For a back reference, update the back reference map and the
- maximum back reference. */
+ /* For a recursion, that's all that is needed. We can now go to
+ the code that handles numerical recursion, applying it to the first
+ group with the given name. */
- cb->backref_map |= (groupnumber < 32)? (1u << groupnumber) : 1;
- if (groupnumber > cb->top_backref)
- cb->top_backref = groupnumber;
- }
+ if (meta == META_RECURSE_BYNAME)
+ {
+ meta_arg = groupnumber;
+ goto HANDLE_NUMERICAL_RECURSION;
}
- /* If the name was not found we have a bad reference. */
+ /* For a back reference, update the back reference map and the
+ maximum back reference. */
- if (groupnumber == 0)
- {
- *errorcodeptr = ERR15;
- cb->erroroffset = offset;
- return 0;
- }
+ cb->backref_map |= (groupnumber < 32)? (1u << groupnumber) : 1;
+ if (groupnumber > cb->top_backref)
+ cb->top_backref = groupnumber;
/* If a back reference name is not duplicated, we can handle it as
a numerical reference. */
- if (!is_dupname)
+ if ((ng->hash_dup & NAMED_GROUP_IS_DUPNAME) == 0)
{
meta_arg = groupnumber;
goto HANDLE_SINGLE_REFERENCE;
@@ -6994,8 +7179,8 @@ for (;; pptr++)
count = 0; /* Values for first pass (avoids compiler warning) */
index = 0;
- if (lengthptr == NULL && !find_dupname_details(name, length, &index,
- &count, errorcodeptr, cb)) return 0;
+ if (lengthptr == NULL && !PRIV(compile_find_dupname_details)(name, length,
+ &index, &count, errorcodeptr, cb)) return 0;
if (firstcuflags == REQ_UNSET) firstcuflags = REQ_NONE;
*code++ = ((options & PCRE2_CASELESS) != 0)? OP_DNREFI : OP_DNREF;
@@ -7279,6 +7464,7 @@ for (;; pptr++)
if (repeat_min > 0 && (repeat_min != 1 || repeat_max != REPEAT_UNLIMITED))
{
int replicate = repeat_min;
+
if (repeat_min == repeat_max) replicate--;
/* In the pre-compile phase, we don't actually do the replication. We
@@ -7288,7 +7474,7 @@ for (;; pptr++)
if (lengthptr != NULL)
{
PCRE2_SIZE delta;
- if (PRIV(ckd_smul)(&delta, replicate, 1 + LINK_SIZE) ||
+ if (PRIV(ckd_smul)(&delta, replicate, (int)length_prevgroup) ||
OFLOW_MAX - *lengthptr < delta)
{
*errorcodeptr = ERR20;
@@ -7296,12 +7482,11 @@ for (;; pptr++)
}
*lengthptr += delta;
}
-
else for (int i = 0; i < replicate; i++)
{
- memcpy(code, previous, CU2BYTES(1 + LINK_SIZE));
+ memcpy(code, previous, CU2BYTES(length_prevgroup));
previous = code;
- code += 1 + LINK_SIZE;
+ code += length_prevgroup;
}
/* If the number of repeats is fixed, we are done. Otherwise, adjust
@@ -7313,18 +7498,21 @@ for (;; pptr++)
}
/* Wrap the recursion call in OP_BRA brackets. */
+ {
+ PCRE2_SIZE length = (lengthptr != NULL) ? 1 + LINK_SIZE : length_prevgroup;
- (void)memmove(previous + 1 + LINK_SIZE, previous, CU2BYTES(1 + LINK_SIZE));
- op_previous = *previous = OP_BRA;
- PUT(previous, 1, 2 + 2*LINK_SIZE);
- previous[2 + 2*LINK_SIZE] = OP_KET;
- PUT(previous, 3 + 2*LINK_SIZE, 2 + 2*LINK_SIZE);
+ (void)memmove(previous + 1 + LINK_SIZE, previous, CU2BYTES(length));
+ op_previous = *previous = OP_BRA;
+ PUT(previous, 1, 1 + LINK_SIZE + length);
+ previous[1 + LINK_SIZE + length] = OP_KET;
+ PUT(previous, 2 + LINK_SIZE + length, 1 + LINK_SIZE + length);
+ }
code += 2 + 2 * LINK_SIZE;
- length_prevgroup = 3 + 3*LINK_SIZE;
+ length_prevgroup += 2 + 2 * LINK_SIZE;
group_return = -1; /* Set "may match empty string" */
/* Now treat as a repeated OP_BRA. */
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
/* If previous was a bracket group, we may have to replicate it in
certain cases. Note that at this point we can encounter only the "basic"
@@ -7664,13 +7852,16 @@ for (;; pptr++)
here because it just makes it horribly messy. */
default:
+
+ /* LCOV_EXCL_START */
if (op_previous >= OP_EODN || op_previous <= OP_WORD_BOUNDARY)
{
PCRE2_DEBUG_UNREACHABLE();
*errorcodeptr = ERR10; /* Not a character type - internal error */
return 0;
}
- else
+ /* LCOV_EXCL_STOP */
+
{
int prop_type, prop_value;
PCRE2_UCHAR *oldcode;
@@ -7993,6 +8184,52 @@ for (;; pptr++)
*code = OP_RECURSE;
PUT(code, 1, meta_arg);
code += 1 + LINK_SIZE;
+ /* Repeat processing requires this information to
+ determine the real length in pre-compile phase. */
+ length_prevgroup = 1 + LINK_SIZE;
+
+ if (META_CODE(pptr[1]) == META_OFFSET ||
+ META_CODE(pptr[1]) == META_CAPTURE_NAME ||
+ META_CODE(pptr[1]) == META_CAPTURE_NUMBER)
+ {
+ recurse_arguments *args;
+
+ if (lengthptr != NULL)
+ {
+ if (!PRIV(compile_parse_recurse_args)(pptr, offset, errorcodeptr, cb))
+ return 0;
+
+ args = (recurse_arguments*)cb->last_data;
+ length_prevgroup += (args->size * (1 + IMM2_SIZE));
+ *lengthptr += (args->size * (1 + IMM2_SIZE));
+ pptr += args->skip_size;
+ }
+ else
+ {
+ uint16_t *current, *end;
+
+ args = (recurse_arguments*)cb->first_data;
+ PCRE2_ASSERT(args != NULL && args->header.type == CDATA_RECURSE_ARGS);
+
+ current = (uint16_t*)(args + 1);
+ end = current + args->size;
+ PCRE2_ASSERT(end > current);
+
+ do
+ {
+ code[0] = OP_CREF;
+ PUT2(code, 1, *current);
+ code += 1 + IMM2_SIZE;
+ }
+ while (++current < end);
+
+ length_prevgroup += (args->size * (1 + IMM2_SIZE));
+ pptr += args->skip_size;
+ cb->first_data = args->header.next;
+ cb->cx->memctl.free(args, cb->cx->memctl.memory_data);
+ }
+ }
+
groupsetfirstcu = FALSE;
cb->had_recurse = TRUE;
if (firstcuflags == REQ_UNSET) firstcuflags = REQ_NONE;
@@ -8117,6 +8354,7 @@ for (;; pptr++)
cb->external_flags |= PCRE2_HASBKC; /* Record */
#if PCRE2_CODE_UNIT_WIDTH == 32
meta_arg = OP_ALLANY;
+ (void)utf; /* Avoid compiler warning. */
#else
if (!utf) meta_arg = OP_ALLANY;
#endif
@@ -8127,11 +8365,15 @@ for (;; pptr++)
if ((options & PCRE2_UCP) != 0 && (xoptions & PCRE2_EXTRA_ASCII_BSW) == 0)
meta_arg = (meta_arg == ESC_B)? OP_NOT_UCP_WORD_BOUNDARY :
OP_UCP_WORD_BOUNDARY;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case ESC_A:
if (cb->max_lookbehind == 0) cb->max_lookbehind = 1;
break;
+
+ case ESC_K:
+ cb->external_flags |= PCRE2_HASBSK; /* Record */
+ break;
}
*code++ = meta_arg;
@@ -8143,12 +8385,14 @@ for (;; pptr++)
META_END is a literal. Otherwise we have a problem. */
default:
+ /* LCOV_EXCL_START */
if (meta >= META_END)
{
PCRE2_DEBUG_UNREACHABLE();
*errorcodeptr = ERR89; /* Internal error - unrecognized. */
return 0;
}
+ /* LCOV_EXCL_STOP */
/* Handle a literal character. We come here by goto in the case of a
32-bit, non-UTF character whose value is greater than META_END. */
@@ -8280,8 +8524,10 @@ for (;; pptr++)
} /* End of big switch */
} /* End of big loop */
+/* LCOV_EXCL_START */
PCRE2_DEBUG_UNREACHABLE(); /* Control should never reach here */
return 0; /* Avoid compiler warnings */
+/* LCOV_EXCL_STOP */
}
@@ -8346,6 +8592,7 @@ if (cb->cx->stack_guard != NULL &&
cb->cx->stack_guard(cb->parens_depth, cb->cx->stack_guard_data))
{
*errorcodeptr= ERR33;
+ cb->erroroffset = 0;
return 0;
}
@@ -8587,8 +8834,10 @@ for (;;)
pptr++;
}
+/* LCOV_EXCL_START */
PCRE2_DEBUG_UNREACHABLE(); /* Control should never reach here */
return 0; /* Avoid compiler warnings */
+/* LCOV_EXCL_STOP */
}
@@ -9060,7 +9309,7 @@ do {
case OP_EXACT:
scode += IMM2_SIZE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_CHAR:
case OP_PLUS:
@@ -9073,7 +9322,7 @@ do {
case OP_EXACTI:
scode += IMM2_SIZE;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_CHARI:
case OP_PLUSI:
@@ -9108,67 +9357,6 @@ return c;
-/*************************************************
-* Add an entry to the name/number table *
-*************************************************/
-
-/* This function is called between compiling passes to add an entry to the
-name/number table, maintaining alphabetical order. Checking for permitted
-and forbidden duplicates has already been done.
-
-Arguments:
- cb the compile data block
- name the name to add
- length the length of the name
- groupno the group number
- tablecount the count of names in the table so far
-
-Returns: nothing
-*/
-
-static void
-add_name_to_table(compile_block *cb, PCRE2_SPTR name, int length,
- unsigned int groupno, uint32_t tablecount)
-{
-uint32_t i;
-PCRE2_UCHAR *slot = cb->name_table;
-
-for (i = 0; i < tablecount; i++)
- {
- int crc = memcmp(name, slot+IMM2_SIZE, CU2BYTES(length));
- if (crc == 0 && slot[IMM2_SIZE+length] != 0)
- crc = -1; /* Current name is a substring */
-
- /* Make space in the table and break the loop for an earlier name. For a
- duplicate or later name, carry on. We do this for duplicates so that in the
- simple case (when ?(| is not used) they are in order of their numbers. In all
- cases they are in the order in which they appear in the pattern. */
-
- if (crc < 0)
- {
- (void)memmove(slot + cb->name_entry_size, slot,
- CU2BYTES((tablecount - i) * cb->name_entry_size));
- break;
- }
-
- /* Continue the loop for a later or duplicate name */
-
- slot += cb->name_entry_size;
- }
-
-PUT2(slot, 0, groupno);
-memcpy(slot + IMM2_SIZE, name, CU2BYTES(length));
-
-/* Add a terminating zero and fill the rest of the slot with zeroes so that
-the memory is all initialized. Otherwise valgrind moans about uninitialized
-memory when saving serialized compiled patterns. */
-
-memset(slot + IMM2_SIZE + length, 0,
- CU2BYTES(cb->name_entry_size - length - IMM2_SIZE));
-}
-
-
-
/*************************************************
* Skip in parsed pattern *
*************************************************/
@@ -9210,13 +9398,14 @@ for (;; pptr++)
if (meta < META_END) continue; /* Literal */
break;
- case META_END:
-
/* The parsed regex is malformed; we have reached the end and did
not find the end of the construct which we are skipping over. */
+ /* LCOV_EXCL_START */
+ case META_END:
PCRE2_DEBUG_UNREACHABLE();
return NULL;
+ /* LCOV_EXCL_STOP */
/* The data for these items is variable in length. */
@@ -9281,7 +9470,9 @@ for (;; pptr++)
pptr += meta_extra_lengths[meta];
}
+/* LCOV_EXCL_START */
PCRE2_UNREACHABLE(); /* Control never reaches here */
+/* LCOV_EXCL_STOP */
}
@@ -9566,36 +9757,30 @@ for (;; pptr++)
case META_BACKREF_BYNAME:
if ((cb->external_options & PCRE2_MATCH_UNSET_BACKREF) != 0)
goto ISNOTFIXED;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case META_RECURSE_BYNAME:
{
- int i;
PCRE2_SPTR name;
BOOL is_dupname = FALSE;
- named_group *ng = cb->named_groups;
+ named_group *ng;
uint32_t meta_code = META_CODE(*pptr);
uint32_t length = *(++pptr);
GETPLUSOFFSET(offset, pptr);
name = cb->start_pattern + offset;
- for (i = 0; i < cb->names_found; i++, ng++)
- {
- if (length == ng->length && PRIV(strncmp)(name, ng->name, length) == 0)
- {
- group = ng->number;
- is_dupname = ng->isdup;
- break;
- }
- }
+ ng = PRIV(compile_find_named_group)(name, length, cb);
- if (group == 0)
+ if (ng == NULL)
{
*errcodeptr = ERR15; /* Non-existent subpattern */
cb->erroroffset = offset;
return -1;
}
+ group = ng->number;
+ is_dupname = (ng->hash_dup & NAMED_GROUP_IS_DUPNAME) != 0;
+
/* A numerical back reference can be fixed length if duplicate capturing
groups are not being used. A non-duplicate named back reference can also
be handled. */
@@ -9621,7 +9806,7 @@ for (;; pptr++)
goto RECURSE_OR_BACKREF_LENGTH;
}
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
/* For groups >= 10 - picking up group twice does no harm. */
/* A true recursion implies not fixed length, but a subroutine call may
@@ -9701,7 +9886,7 @@ for (;; pptr++)
case META_CAPTURE:
group = META_DATA(*pptr);
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case META_ATOMIC:
case META_NOCAPTURE:
@@ -9748,7 +9933,7 @@ for (;; pptr++)
else itemlength = (max - 1) * lastitemlength;
break;
}
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
/* Any other item means this branch does not have a fixed length. */
@@ -9783,10 +9968,12 @@ EXIT:
*minptr = branchminlength;
return branchlength;
+/* LCOV_EXCL_START */
PARSED_SKIP_FAILED:
PCRE2_DEBUG_UNREACHABLE();
*errcodeptr = ERR90; /* Unhandled META code - internal error */
return -1;
+/* LCOV_EXCL_STOP */
}
@@ -9922,15 +10109,16 @@ for (; *pptr != META_END; pptr++)
switch (META_CODE(*pptr))
{
- default:
-
/* The following erroroffset is a bogus but safe value. This branch should
be avoided by providing a proper implementation for all supported cases
below. */
+ /* LCOV_EXCL_START */
+ default:
PCRE2_DEBUG_UNREACHABLE();
cb->erroroffset = 0;
return ERR70; /* Unrecognized meta code */
+ /* LCOV_EXCL_STOP */
case META_ESCAPE:
if (*pptr - META_ESCAPE == ESC_P || *pptr - META_ESCAPE == ESC_p)
@@ -10021,8 +10209,8 @@ for (; *pptr != META_END; pptr++)
case META_BIGVALUE:
case META_POSIX:
case META_POSIX_NEG:
- case META_SCS_NAME:
- case META_SCS_NUMBER:
+ case META_CAPTURE_NAME:
+ case META_CAPTURE_NUMBER:
pptr += 1;
break;
@@ -10093,8 +10281,9 @@ pcre2_real_code *re = NULL; /* What we will return */
compile_block cb; /* "Static" compile-time data */
const uint8_t *tables; /* Char tables base pointer */
+PCRE2_UCHAR null_str[1] = { 0xcd }; /* Dummy for handling null inputs */
PCRE2_UCHAR *code; /* Current pointer in compiled code */
-PCRE2_UCHAR * codestart; /* Start of compiled code */
+PCRE2_UCHAR *codestart; /* Start of compiled code */
PCRE2_SPTR ptr; /* Current pointer in pattern */
uint32_t *pptr; /* Current pointer in parsed pattern */
@@ -10141,7 +10330,16 @@ PCRE2_UCHAR *cworkspace = (PCRE2_UCHAR *)c16workspace;
/* There must be error code and offset pointers. */
-if (errorptr == NULL || erroroffset == NULL) return NULL;
+if (errorptr == NULL)
+ {
+ if (erroroffset != NULL) *erroroffset = 0;
+ return NULL;
+ }
+if (erroroffset == NULL)
+ {
+ if (errorptr != NULL) *errorptr = ERR120;
+ return NULL;
+ }
*errorptr = ERR0;
*erroroffset = 0;
@@ -10149,7 +10347,9 @@ if (errorptr == NULL || erroroffset == NULL) return NULL;
if (pattern == NULL)
{
- if (patlen == 0) pattern = (PCRE2_SPTR)""; else
+ if (patlen == 0)
+ pattern = null_str;
+ else
{
*errorptr = ERR16;
return NULL;
@@ -10245,9 +10445,9 @@ cb.start_code = cworkspace;
cb.start_pattern = pattern;
cb.start_workspace = cworkspace;
cb.workspace_size = COMPILE_WORK_SIZE;
+cb.first_data = NULL;
+cb.last_data = NULL;
#ifdef SUPPORT_WIDE_CHARS
-cb.cranges = NULL;
-cb.next_cranges = NULL;
cb.char_lists_size = 0;
#endif
@@ -10341,6 +10541,7 @@ if ((options & PCRE2_LITERAL) == 0)
{
errorcode = ERR60;
ptr += pp;
+ utf = FALSE; /* Used by HAD_EARLY_ERROR */
goto HAD_EARLY_ERROR;
}
if (p->type == PSO_LIMH) limit_heap = c;
@@ -10373,11 +10574,13 @@ if ((options & PCRE2_LITERAL) == 0)
break;
+ /* LCOV_EXCL_START */
default:
/* All values in the enum need an explicit entry for this switch
but until a better way to prevent coding mistakes is invented keep
a catch all that triggers a debug build assert as a failsafe */
PCRE2_DEBUG_UNREACHABLE();
+ /* LCOV_EXCL_STOP */
}
break; /* Out of the table scan loop */
}
@@ -10500,10 +10703,12 @@ switch(newline)
cb.nltype = NLTYPE_ANYCRLF;
break;
+ /* LCOV_EXCL_START */
default:
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR56;
goto HAD_EARLY_ERROR;
+ /* LCOV_EXCL_STOP */
}
/* Pre-scan the pattern to do two things: (1) Discover the named groups and
@@ -10632,6 +10837,7 @@ if (length > MAX_PATTERN_SIZE)
#endif
{
errorcode = ERR20;
+ cb.erroroffset = 0;
goto HAD_CB_ERROR;
}
@@ -10660,6 +10866,7 @@ re_blocksize += CU2BYTES(length);
if (re_blocksize > ccontext->max_pattern_compiled_length)
{
errorcode = ERR101;
+ cb.erroroffset = 0;
goto HAD_CB_ERROR;
}
@@ -10669,6 +10876,7 @@ re = (pcre2_real_code *)
if (re == NULL)
{
errorcode = ERR21;
+ cb.erroroffset = 0;
goto HAD_CB_ERROR;
}
@@ -10734,8 +10942,14 @@ created in the pre-pass. */
if (cb.names_found > 0)
{
named_group *ng = cb.named_groups;
+ uint32_t tablecount = 0;
+
+ /* Length 0 represents duplicates, and they have already been handled. */
for (i = 0; i < cb.names_found; i++, ng++)
- add_name_to_table(&cb, ng->name, ng->length, ng->number, i);
+ if (ng->length > 0)
+ tablecount = PRIV(compile_add_name_to_table)(&cb, ng, tablecount);
+
+ PCRE2_ASSERT(tablecount == cb.names_found);
}
/* Set up a starting, non-extracting bracket, then compile the expression. On
@@ -10767,18 +10981,20 @@ memory as unaddressable, so that any out-of-bound reads can be detected. */
*code++ = OP_END;
usedlength = code - codestart;
+/* LCOV_EXCL_START */
if (usedlength > length)
{
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR23; /* Overflow of code block - internal error */
+ cb.erroroffset = 0;
+ goto HAD_CB_ERROR;
}
-else
- {
- re->blocksize -= CU2BYTES(length - usedlength);
+/* LCOV_EXCL_STOP */
+
+re->blocksize -= CU2BYTES(length - usedlength);
#ifdef SUPPORT_VALGRIND
- VALGRIND_MAKE_MEM_NOACCESS(code, CU2BYTES(length - usedlength));
+VALGRIND_MAKE_MEM_NOACCESS(code, CU2BYTES(length - usedlength));
#endif
- }
/* Scan the pattern for recursion/subroutine calls and convert the group
numbers into offsets. Maintain a small cache so that repeated groups containing
@@ -10823,12 +11039,15 @@ if (errorcode == 0 && cb.had_recurse)
if (rgroup == NULL)
{
rgroup = PRIV(find_bracket)(search_from, utf, groupnumber);
+ /* LCOV_EXCL_START */
if (rgroup == NULL)
{
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR53;
break;
}
+ /* LCOV_EXCL_STOP */
+
if (--start < 0) start = RSCAN_CACHE_SIZE - 1;
rc[start].groupnumber = groupnumber;
rc[start].group = rgroup;
@@ -10858,11 +11077,15 @@ function call. */
if (errorcode == 0 && (optim_flags & PCRE2_OPTIM_AUTO_POSSESS) != 0)
{
PCRE2_UCHAR *temp = (PCRE2_UCHAR *)codestart;
- if (PRIV(auto_possessify)(temp, &cb) != 0)
+ int possessify_rc = PRIV(auto_possessify)(temp, &cb);
+ /* LCOV_EXCL_START */
+ if (possessify_rc != 0)
{
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR80;
+ cb.erroroffset = 0;
}
+ /* LCOV_EXCL_STOP */
}
/* Failed to compile, or error while post-processing. */
@@ -10892,6 +11115,7 @@ unit. */
if ((optim_flags & PCRE2_OPTIM_START_OPTIMIZE) != 0)
{
int minminlength = 0; /* For minimal minlength from first/required CU */
+ int study_rc;
/* If we do not have a first code unit, see if there is one that is asserted
(these are not saved during the compile because they can cause conflicts with
@@ -11015,12 +11239,16 @@ if ((optim_flags & PCRE2_OPTIM_START_OPTIMIZE) != 0)
/* Study the compiled pattern to set up information such as a bitmap of
starting code units and a minimum matching length. */
- if (PRIV(study)(re) != 0)
+ study_rc = PRIV(study)(re);
+ /* LCOV_EXCL_START */
+ if (study_rc != 0)
{
PCRE2_DEBUG_UNREACHABLE();
errorcode = ERR31;
+ cb.erroroffset = 0;
goto HAD_CB_ERROR;
}
+ /* LCOV_EXCL_STOP */
/* If study() set a bitmap of starting code units, it implies a minimum
length of at least one. */
@@ -11041,7 +11269,8 @@ groups if a larger one had to be obtained, and likewise the group information
vector. */
#ifdef SUPPORT_UNICODE
-PCRE2_ASSERT(cb.cranges == NULL);
+/* All items must be freed. */
+PCRE2_ASSERT(cb.first_data == NULL);
#endif
EXIT:
@@ -11067,8 +11296,22 @@ HAD_CB_ERROR:
ptr = pattern + cb.erroroffset;
HAD_EARLY_ERROR:
-PCRE2_ASSERT(ptr >= pattern); /* Ensure we don't return invalid erroroffset */
+/* Ensure we don't return out-of-range erroroffset. */
+PCRE2_ASSERT(ptr >= pattern);
PCRE2_ASSERT(ptr <= (pattern + patlen));
+/* Ensure that the erroroffset never slices a UTF-encoded character in half.
+If the input is invalid, then we return an offset just before the first invalid
+character, so the text to the left of the offset must always be valid. */
+#if defined PCRE2_DEBUG && defined SUPPORT_UNICODE
+if (ptr > pattern && utf)
+ {
+ PCRE2_SPTR prev = ptr - 1;
+ PCRE2_SIZE dummyoffset;
+ BACKCHAR(prev);
+ PCRE2_ASSERT(prev >= pattern);
+ PCRE2_ASSERT(PRIV(valid_utf)(prev, ptr - prev, &dummyoffset) == 0);
+ }
+#endif
*erroroffset = ptr - pattern;
HAD_ERROR:
@@ -11076,19 +11319,18 @@ HAD_ERROR:
pcre2_code_free(re);
re = NULL;
-#ifdef SUPPORT_WIDE_CHARS
-if (cb.cranges != NULL)
+if (cb.first_data != NULL)
{
- class_ranges* cranges = cb.cranges;
+ compile_data* current_data = cb.first_data;
do
{
- class_ranges* next_cranges = cranges->next;
- cb.cx->memctl.free(cranges, cb.cx->memctl.memory_data);
- cranges = next_cranges;
+ compile_data* next_data = current_data->next;
+ cb.cx->memctl.free(current_data, cb.cx->memctl.memory_data);
+ current_data = next_data;
}
- while (cranges != NULL);
+ while (current_data != NULL);
}
-#endif
+
goto EXIT;
}
diff --git a/src/pcre2_compile.h b/src/pcre2_compile.h
index c8bf610..331ca3a 100644
--- a/src/pcre2_compile.h
+++ b/src/pcre2_compile.h
@@ -51,18 +51,18 @@ pcre2.h.in must be updated - their values are exactly 100 greater than these
values. */
enum { ERR0 = COMPILE_ERROR_BASE,
- ERR1, ERR2, ERR3, ERR4, ERR5, ERR6, ERR7, ERR8, ERR9, ERR10,
- ERR11, ERR12, ERR13, ERR14, ERR15, ERR16, ERR17, ERR18, ERR19, ERR20,
- ERR21, ERR22, ERR23, ERR24, ERR25, ERR26, ERR27, ERR28, ERR29, ERR30,
- ERR31, ERR32, ERR33, ERR34, ERR35, ERR36, ERR37, ERR38, ERR39, ERR40,
- ERR41, ERR42, ERR43, ERR44, ERR45, ERR46, ERR47, ERR48, ERR49, ERR50,
- ERR51, ERR52, ERR53, ERR54, ERR55, ERR56, ERR57, ERR58, ERR59, ERR60,
- ERR61, ERR62, ERR63, ERR64, ERR65, ERR66, ERR67, ERR68, ERR69, ERR70,
- ERR71, ERR72, ERR73, ERR74, ERR75, ERR76, ERR77, ERR78, ERR79, ERR80,
- ERR81, ERR82, ERR83, ERR84, ERR85, ERR86, ERR87, ERR88, ERR89, ERR90,
- ERR91, ERR92, ERR93, ERR94, ERR95, ERR96, ERR97, ERR98, ERR99, ERR100,
- ERR101,ERR102,ERR103,ERR104,ERR105,ERR106,ERR107,ERR108,ERR109,ERR110,
- ERR111,ERR112,ERR113,ERR114,ERR115,ERR116 };
+ ERR1, ERR2, ERR3, ERR4, ERR5, ERR6, ERR7, ERR8, ERR9, ERR10,
+ ERR11, ERR12, ERR13, ERR14, ERR15, ERR16, ERR17, ERR18, ERR19, ERR20,
+ ERR21, ERR22, ERR23, ERR24, ERR25, ERR26, ERR27, ERR28, ERR29, ERR30,
+ ERR31, ERR32, ERR33, ERR34, ERR35, ERR36, ERR37, ERR38, ERR39, ERR40,
+ ERR41, ERR42, ERR43, ERR44, ERR45, ERR46, ERR47, ERR48, ERR49, ERR50,
+ ERR51, ERR52, ERR53, ERR54, ERR55, ERR56, ERR57, ERR58, ERR59, ERR60,
+ ERR61, ERR62, ERR63, ERR64, ERR65, ERR66, ERR67, ERR68, ERR69, ERR70,
+ ERR71, ERR72, ERR73, ERR74, ERR75, ERR76, ERR77, ERR78, ERR79, ERR80,
+ ERR81, ERR82, ERR83, ERR84, ERR85, ERR86, ERR87, ERR88, ERR89, ERR90,
+ ERR91, ERR92, ERR93, ERR94, ERR95, ERR96, ERR97, ERR98, ERR99, ERR100,
+ ERR101, ERR102, ERR103, ERR104, ERR105, ERR106, ERR107, ERR108, ERR109, ERR110,
+ ERR111, ERR112, ERR113, ERR114, ERR115, ERR116, ERR117, ERR118, ERR119, ERR120 };
/* Code values for parsed patterns, which are stored in a vector of 32-bit
unsigned ints. Values less than META_END are literal data values. The coding
@@ -96,11 +96,11 @@ code (meta_extra_lengths) must be updated to remain in step. */
#define META_COND_RNAME 0x80130000u /* (?(R&name)... */
#define META_COND_RNUMBER 0x80140000u /* (?(Rdigits)... */
#define META_COND_VERSION 0x80150000u /* (?(VERSIONx.y)... */
-#define META_OFFSET 0x80160000u /* Setting offset for various
- META codes (e.g. META_SCS_NAME) */
+#define META_OFFSET 0x80160000u /* Setting offset for various META
+ codes (e.g. META_CAPTURE_NAME) */
#define META_SCS 0x80170000u /* (*scan_substring:... */
-#define META_SCS_NAME 0x80180000u /* Next of scan_substring */
-#define META_SCS_NUMBER 0x80190000u /* Next digits of scan_substring */
+#define META_CAPTURE_NAME 0x80180000u /* Next in capture lists */
+#define META_CAPTURE_NUMBER 0x80190000u /* Next digits in capture lists */
#define META_DOLLAR 0x801a0000u /* $ metacharacter */
#define META_DOT 0x801b0000u /* . metacharacter */
#define META_ESCAPE 0x801c0000u /* \d and friends */
@@ -186,6 +186,36 @@ therefore no need for it to have a length entry, so use a high value. */
#define META_DATA(x) (x & 0x0000ffffu)
#define META_DIFF(x,y) ((x-y)>>16)
+/* Macros to store and retrieve a PCRE2_SIZE value in the parsed pattern, which
+consists of uint32_t elements. Assume that if uint32_t can't hold it, two of
+them will be able to (i.e. assume a 64-bit world). */
+
+#if PCRE2_SIZE_MAX <= UINT32_MAX
+#define PUTOFFSET(s,p) *p++ = s
+#define GETOFFSET(s,p) s = *p++
+#define GETPLUSOFFSET(s,p) s = *(++p)
+#define READPLUSOFFSET(s,p) s = p[1]
+#define SKIPOFFSET(p) p++
+#define SIZEOFFSET 1
+#else
+#define PUTOFFSET(s,p) \
+ { *p++ = (uint32_t)(s >> 32); *p++ = (uint32_t)(s & 0xffffffff); }
+#define GETOFFSET(s,p) \
+ { s = ((PCRE2_SIZE)p[0] << 32) | (PCRE2_SIZE)p[1]; p += 2; }
+#define GETPLUSOFFSET(s,p) \
+ { s = ((PCRE2_SIZE)p[1] << 32) | (PCRE2_SIZE)p[2]; p += 2; }
+#define READPLUSOFFSET(s,p) \
+ { s = ((PCRE2_SIZE)p[1] << 32) | (PCRE2_SIZE)p[2]; }
+#define SKIPOFFSET(p) p += 2
+#define SIZEOFFSET 2
+#endif
+
+#ifdef PCRE2_DEBUG
+/* Compile data types. */
+#define CDATA_RECURSE_ARGS 0 /* Argument list for recurse */
+#define CDATA_CRANGE 1 /* Character range list */
+#endif
+
/* Extended class management flags. */
#define CLASS_IS_ECLASS 0x1
@@ -236,10 +266,16 @@ typedef struct {
/* Macros for the definitions below, to prevent name collisions. */
-#define _pcre2_posix_class_maps PCRE2_SUFFIX(_pcre2_posix_class_maps)
-#define _pcre2_update_classbits PCRE2_SUFFIX(_pcre2_update_classbits_)
-#define _pcre2_compile_class_nested PCRE2_SUFFIX(_pcre2_compile_class_nested_)
-#define _pcre2_compile_class_not_nested PCRE2_SUFFIX(_pcre2_compile_class_not_nested_)
+#define _pcre2_posix_class_maps PCRE2_SUFFIX(_pcre2_posix_class_maps)
+#define _pcre2_update_classbits PCRE2_SUFFIX(_pcre2_update_classbits_)
+#define _pcre2_compile_class_nested PCRE2_SUFFIX(_pcre2_compile_class_nested_)
+#define _pcre2_compile_class_not_nested PCRE2_SUFFIX(_pcre2_compile_class_not_nested_)
+#define _pcre2_compile_get_hash_from_name PCRE2_SUFFIX(_pcre2_compile_get_hash_from_name)
+#define _pcre2_compile_find_named_group PCRE2_SUFFIX(_pcre2_compile_find_named_group)
+#define _pcre2_compile_find_dupname_details PCRE2_SUFFIX(_pcre2_compile_find_dupname_details)
+#define _pcre2_compile_add_name_to_table PCRE2_SUFFIX(_pcre2_compile_add_name_to_table)
+#define _pcre2_compile_parse_scan_substr_args PCRE2_SUFFIX(_pcre2_compile_parse_scan_substr_args)
+#define _pcre2_compile_parse_recurse_args PCRE2_SUFFIX(_pcre2_compile_parse_recurse_args)
/* Indices of the POSIX classes in posix_names, posix_name_lengths,
@@ -253,6 +289,14 @@ posix_class_maps, and posix_substitutes. They must be kept in sync. */
extern const int PRIV(posix_class_maps)[];
+/* Defines for hash_dup member in named_group structure. */
+
+#define NAMED_GROUP_HASH_MASK ((uint16_t)0x7fff)
+#define NAMED_GROUP_IS_DUPNAME ((uint16_t)0x8000)
+
+#define NAMED_GROUP_GET_HASH(ng) ((ng)->hash_dup & NAMED_GROUP_HASH_MASK)
+
+/* Exported functions from pcre2_compile_class.c file: */
/* Set bits in classbits according to the property type */
@@ -275,6 +319,38 @@ BOOL PRIV(compile_class_nested)(uint32_t options, uint32_t xoptions,
uint32_t **pptr, PCRE2_UCHAR **pcode, int *errorcodeptr,
compile_block *cb, PCRE2_SIZE *lengthptr);
+/* Exported functions from pcre2_compile_cgroup.c file: */
+
+/* Compute hash from a capture name. */
+
+uint16_t PRIV(compile_get_hash_from_name)(PCRE2_SPTR name, uint32_t length);
+
+/* Get the descriptor of a known named capture. */
+
+named_group *PRIV(compile_find_named_group)(PCRE2_SPTR name,
+ uint32_t length, compile_block *cb);
+
+/* Add entires to name table in alphabetical order. */
+
+uint32_t PRIV(compile_add_name_to_table)(compile_block *cb,
+ named_group *ng, uint32_t tablecount);
+
+/* Searches the properties of duplicated names, and returns them
+in indexptr and countptr. */
+
+BOOL PRIV(compile_find_dupname_details)(PCRE2_SPTR name, uint32_t length,
+ int *indexptr, int *countptr, int *errorcodeptr, compile_block *cb);
+
+/* Parse the arguments of recurse operations. */
+
+uint32_t * PRIV(compile_parse_scan_substr_args)(uint32_t *pptr,
+ int *errorcodeptr, compile_block *cb, PCRE2_SIZE *lengthptr);
+
+/* Parse the arguments of recurse operations. */
+
+BOOL PRIV(compile_parse_recurse_args)(uint32_t *pptr_start,
+ PCRE2_SIZE offset, int *errorcodeptr, compile_block *cb);
+
#endif /* PCRE2_COMPILE_H_IDEMPOTENT_GUARD */
/* End of pcre2_compile.h */
diff --git a/src/pcre2_compile_cgroup.c b/src/pcre2_compile_cgroup.c
new file mode 100644
index 0000000..ee0cd06
--- /dev/null
+++ b/src/pcre2_compile_cgroup.c
@@ -0,0 +1,632 @@
+/*************************************************
+* Perl-Compatible Regular Expressions *
+*************************************************/
+
+/* PCRE is a library of functions to support regular expressions whose syntax
+and semantics are as close as possible to those of the Perl 5 language.
+
+ Written by Philip Hazel
+ Original API code Copyright (c) 1997-2012 University of Cambridge
+ New API code Copyright (c) 2016-2024 University of Cambridge
+
+-----------------------------------------------------------------------------
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+ * Redistributions of source code must retain the above copyright notice,
+ this list of conditions and the following disclaimer.
+
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+
+ * Neither the name of the University of Cambridge nor the names of its
+ contributors may be used to endorse or promote products derived from
+ this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+-----------------------------------------------------------------------------
+*/
+
+
+#include "pcre2_compile.h"
+
+/*************************************************
+* Compute the hash code from a capture name *
+*************************************************/
+
+/* This function returns with a simple hash code
+computed from the name of a capture group.
+
+Arguments:
+ name name of the capture group
+ length the length of the name
+
+Returns: hash code
+*/
+
+uint16_t
+PRIV(compile_get_hash_from_name)(PCRE2_SPTR name, uint32_t length)
+{
+uint16_t hash;
+
+PCRE2_ASSERT(length > 0);
+
+hash = (uint16_t)((name[0] & 0x7f) | ((name[length - 1] & 0xff) << 7));
+PCRE2_ASSERT(hash <= NAMED_GROUP_HASH_MASK);
+return hash;
+}
+
+
+/*************************************************
+* Get the descriptor of a known named capture *
+*************************************************/
+
+/* This function returns the descriptor in the
+named group list of a known capture group.
+
+Arguments:
+ name name of the capture group
+ length the length of the name
+
+Returns: pointer to the descriptor when found,
+ NULL otherwise
+ */
+
+named_group *
+PRIV(compile_find_named_group)(PCRE2_SPTR name,
+ uint32_t length, compile_block *cb)
+{
+uint16_t hash = PRIV(compile_get_hash_from_name)(name, length);
+named_group *ng;
+named_group *end = cb->named_groups + cb->names_found;
+
+for (ng = cb->named_groups; ng < end; ng++)
+ if (length == ng->length && hash == NAMED_GROUP_GET_HASH(ng) &&
+ PRIV(strncmp)(name, ng->name, length) == 0) return ng;
+
+return NULL;
+}
+
+
+/*************************************************
+* Add an entry to the name/number table *
+*************************************************/
+
+/* This function is called between compiling passes to add an entry to the
+name/number table, maintaining alphabetical order. Checking for permitted
+and forbidden duplicates has already been done.
+
+Arguments:
+ cb the compile data block
+ nb named group entry
+ tablecount the count of names in the table so far
+
+Returns: new tablecount
+*/
+
+uint32_t
+PRIV(compile_add_name_to_table)(compile_block *cb,
+ named_group *ng, uint32_t tablecount)
+{
+uint32_t i;
+PCRE2_SPTR name = ng->name;
+int length = ng->length;
+uint32_t duplicate_count = 1;
+
+PCRE2_UCHAR *slot = cb->name_table;
+
+PCRE2_ASSERT(length > 0);
+
+if ((ng->hash_dup & NAMED_GROUP_IS_DUPNAME) != 0)
+ {
+ named_group *ng_it;
+ named_group *end = cb->named_groups + cb->names_found;
+
+ for (ng_it = ng + 1; ng_it < end; ng_it++)
+ if (ng_it->name == name) duplicate_count++;
+ }
+
+for (i = 0; i < tablecount; i++)
+ {
+ int crc = memcmp(name, slot + IMM2_SIZE, CU2BYTES(length));
+ if (crc == 0 && slot[IMM2_SIZE + length] != 0)
+ crc = -1; /* Current name is a substring */
+
+ /* Make space in the table and break the loop for an earlier name. For a
+ duplicate or later name, carry on. We do this for duplicates so that in the
+ simple case (when ?(| is not used) they are in order of their numbers. In all
+ cases they are in the order in which they appear in the pattern. */
+
+ if (crc < 0)
+ {
+ (void)memmove(slot + cb->name_entry_size * duplicate_count, slot,
+ CU2BYTES((tablecount - i) * cb->name_entry_size));
+ break;
+ }
+
+ /* Continue the loop for a later or duplicate name */
+
+ slot += cb->name_entry_size;
+ }
+
+tablecount += duplicate_count;
+
+while (TRUE)
+ {
+ PUT2(slot, 0, ng->number);
+ memcpy(slot + IMM2_SIZE, name, CU2BYTES(length));
+
+ /* Add a terminating zero and fill the rest of the slot with zeroes so that
+ the memory is all initialized. Otherwise valgrind moans about uninitialized
+ memory when saving serialized compiled patterns. */
+
+ memset(slot + IMM2_SIZE + length, 0,
+ CU2BYTES(cb->name_entry_size - length - IMM2_SIZE));
+
+ if (--duplicate_count == 0) break;
+
+ while (TRUE)
+ {
+ ++ng;
+ if (ng->name == name) break;
+ }
+
+ slot += cb->name_entry_size;
+ }
+
+return tablecount;
+}
+
+
+/*************************************************
+* Find details of duplicate group names *
+*************************************************/
+
+/* This is called from compile_branch() when it needs to know the index and
+count of duplicates in the names table when processing named backreferences,
+either directly, or as conditions.
+
+Arguments:
+ name points to the name
+ length the length of the name
+ indexptr where to put the index
+ countptr where to put the count of duplicates
+ errorcodeptr where to put an error code
+ cb the compile block
+
+Returns: TRUE if OK, FALSE if not, error code set
+*/
+
+BOOL
+PRIV(compile_find_dupname_details)(PCRE2_SPTR name, uint32_t length,
+ int *indexptr, int *countptr, int *errorcodeptr, compile_block *cb)
+{
+uint32_t i, groupnumber;
+int count;
+PCRE2_UCHAR *slot = cb->name_table;
+
+/* Find the first entry in the table */
+
+for (i = 0; i < cb->names_found; i++)
+ {
+ if (PRIV(strncmp)(name, slot + IMM2_SIZE, length) == 0 &&
+ slot[IMM2_SIZE + length] == 0) break;
+ slot += cb->name_entry_size;
+ }
+
+/* This should not occur, because this function is called only when we know we
+have duplicate names. Give an internal error. */
+
+/* LCOV_EXCL_START */
+if (i >= cb->names_found)
+ {
+ PCRE2_DEBUG_UNREACHABLE();
+ *errorcodeptr = ERR53;
+ cb->erroroffset = name - cb->start_pattern;
+ return FALSE;
+ }
+/* LCOV_EXCL_STOP */
+
+/* Record the index and then see how many duplicates there are, updating the
+backref map and maximum back reference as we do. */
+
+*indexptr = i;
+count = 0;
+
+for (;;)
+ {
+ count++;
+ groupnumber = GET2(slot, 0);
+ cb->backref_map |= (groupnumber < 32)? (1u << groupnumber) : 1;
+ if (groupnumber > cb->top_backref) cb->top_backref = groupnumber;
+ if (++i >= cb->names_found) break;
+ slot += cb->name_entry_size;
+ if (PRIV(strncmp)(name, slot + IMM2_SIZE, length) != 0 ||
+ (slot + IMM2_SIZE)[length] != 0) break;
+ }
+
+*countptr = count;
+return TRUE;
+}
+
+
+/* Process the capture list of scan substring and recurse
+operations. Since at least one argument must be present,
+a 0 return value represents error. */
+
+static size_t
+PRIV(compile_process_capture_list)(uint32_t *pptr, PCRE2_SIZE offset,
+ int *errorcodeptr, compile_block *cb)
+{
+size_t i, size = 0;
+named_group *ng;
+PCRE2_SPTR name;
+uint32_t length;
+named_group *end = cb->named_groups + cb->names_found;
+
+while (TRUE)
+ {
+ ++pptr;
+
+ switch (META_CODE(*pptr))
+ {
+ case META_OFFSET:
+ GETPLUSOFFSET(offset, pptr);
+ continue;
+
+ case META_CAPTURE_NAME:
+ offset += META_DATA(*pptr);
+ length = *(++pptr);
+ name = cb->start_pattern + offset;
+
+ ng = PRIV(compile_find_named_group)(name, length, cb);
+
+ if (ng == NULL)
+ {
+ *errorcodeptr = ERR15;
+ cb->erroroffset = offset;
+ return 0;
+ }
+
+ if ((ng->hash_dup & NAMED_GROUP_IS_DUPNAME) == 0)
+ {
+ pptr[-1] = META_CAPTURE_NUMBER;
+ pptr[0] = ng->number;
+ size++;
+ continue;
+ }
+
+ /* Remains only for duplicated names. */
+ pptr[-1] = META_CAPTURE_NAME;
+ pptr[0] = (uint32_t)(ng - cb->named_groups);
+ size++;
+ name = ng->name;
+
+ while (++ng < end)
+ if (ng->name == name) size++;
+ continue;
+
+ case META_CAPTURE_NUMBER:
+ offset += META_DATA(*pptr);
+
+ i = *(++pptr);
+ if (i > cb->bracount)
+ {
+ *errorcodeptr = ERR15;
+ cb->erroroffset = offset;
+ return 0;
+ }
+ if (i > cb->top_backref) cb->top_backref = (uint16_t)i;
+ size++;
+ continue;
+
+ default:
+ break;
+ }
+
+ PCRE2_ASSERT(size > 0);
+ return size;
+ }
+}
+
+
+/*******************************************************
+* Parse the arguments of scan substring operations *
+********************************************************/
+
+/* This function parses the arguments of scan substring operations.
+
+Arguments:
+ pptr_start points to the current parsed pattern pointer
+ offset argument starting offset in the pattern
+ errorcodeptr where to put an error code
+ cb the compile block
+ lengthptr NULL during the real compile phase
+ points to length accumulator during pre-compile phase
+
+Returns: TRUE if OK, FALSE if not, error code set
+*/
+
+uint32_t *
+PRIV(compile_parse_scan_substr_args)(uint32_t *pptr,
+ int *errorcodeptr, compile_block *cb, PCRE2_SIZE *lengthptr)
+{
+uint8_t *captures;
+uint8_t *capture_ptr;
+uint8_t bit;
+PCRE2_SPTR name;
+named_group *ng;
+named_group *end = cb->named_groups + cb->names_found;
+BOOL all_found;
+size_t size;
+
+PCRE2_ASSERT(*pptr == META_OFFSET);
+if (PRIV(compile_process_capture_list)(pptr - 1, 0, errorcodeptr, cb) == 0)
+ return NULL;
+
+/* Align to bytes. Since the highest capture can
+be equal to bracount, +1 is added before the aligning. */
+size = (cb->bracount + 1 + 7) >> 3;
+captures = (uint8_t*)cb->cx->memctl.malloc(size, cb->cx->memctl.memory_data);
+if (captures == NULL)
+ {
+ *errorcodeptr = ERR21;
+ READPLUSOFFSET(cb->erroroffset, pptr);
+ return NULL;
+ }
+
+memset(captures, 0, size);
+
+while (TRUE)
+ {
+ switch (META_CODE(*pptr))
+ {
+ case META_OFFSET:
+ pptr++;
+ SKIPOFFSET(pptr);
+ continue;
+
+ case META_CAPTURE_NAME:
+ ng = cb->named_groups + pptr[1];
+ PCRE2_ASSERT((ng->hash_dup & NAMED_GROUP_IS_DUPNAME) != 0);
+ pptr += 2;
+ name = ng->name;
+
+ all_found = TRUE;
+ do
+ {
+ if (ng->name != name) continue;
+
+ capture_ptr = captures + (ng->number >> 3);
+ PCRE2_ASSERT(capture_ptr < captures + size);
+ bit = (uint8_t)(1 << (ng->number & 0x7));
+
+ if ((*capture_ptr & bit) == 0)
+ {
+ *capture_ptr |= bit;
+ all_found = FALSE;
+ }
+ }
+ while (++ng < end);
+
+ if (!all_found)
+ {
+ *lengthptr += 1 + 2 * IMM2_SIZE;
+ continue;
+ }
+
+ pptr[-2] = META_CAPTURE_NUMBER;
+ pptr[-1] = 0;
+ continue;
+
+ case META_CAPTURE_NUMBER:
+ pptr += 2;
+
+ capture_ptr = captures + (pptr[-1] >> 3);
+ PCRE2_ASSERT(capture_ptr < captures + size);
+ bit = (uint8_t)(1 << (pptr[-1] & 0x7));
+
+ if ((*capture_ptr & bit) != 0)
+ {
+ pptr[-1] = 0;
+ continue;
+ }
+
+ *capture_ptr |= bit;
+ *lengthptr += 1 + IMM2_SIZE;
+ continue;
+
+ default:
+ break;
+ }
+
+ break;
+ }
+
+cb->cx->memctl.free(captures, cb->cx->memctl.memory_data);
+return pptr - 1;
+}
+
+
+/* Implement heapsort heapify algorithm. */
+
+static void do_heapify_u16(uint16_t *captures, size_t size, size_t i)
+{
+size_t max;
+size_t left;
+size_t right;
+uint16_t tmp;
+
+while (TRUE)
+ {
+ max = i;
+ left = (i << 1) + 1;
+ right = left + 1;
+
+ if (left < size && captures[left] > captures[max]) max = left;
+ if (right < size && captures[right] > captures[max]) max = right;
+ if (i == max) return;
+
+ tmp = captures[i];
+ captures[i] = captures[max];
+ captures[max] = tmp;
+ i = max;
+ }
+}
+
+
+/*************************************************
+* Parse the arguments of recurse operations *
+*************************************************/
+
+/* This function parses the arguments of recurse operations.
+
+Arguments:
+ pptr_start the current parsed pattern pointer
+ offset argument starting offset in the pattern
+ errorcodeptr where to put an error code
+ cb the compile block
+ lengthptr NULL during the real compile phase
+ points to length accumulator during pre-compile phase
+
+Returns: TRUE if OK, FALSE if not, error code set
+*/
+
+BOOL
+PRIV(compile_parse_recurse_args)(uint32_t *pptr_start,
+ PCRE2_SIZE offset, int *errorcodeptr, compile_block *cb)
+{
+uint32_t *pptr = pptr_start;
+size_t i, size;
+PCRE2_SPTR name;
+named_group *ng;
+named_group *end = cb->named_groups + cb->names_found;
+recurse_arguments *args;
+uint16_t *captures;
+uint16_t *current;
+uint16_t *captures_end;
+uint16_t tmp;
+
+/* Process all arguments, compute the required size. */
+
+size = PRIV(compile_process_capture_list)(pptr, offset, errorcodeptr, cb);
+if (size == 0) return FALSE;
+
+args = cb->cx->memctl.malloc(
+ sizeof(recurse_arguments) + size * sizeof(uint16_t), cb->cx->memctl.memory_data);
+
+if (args == NULL)
+ {
+ *errorcodeptr = ERR21;
+ cb->erroroffset = offset;
+ return FALSE;
+ }
+
+args->header.next = NULL;
+#ifdef PCRE2_DEBUG
+args->header.type = CDATA_RECURSE_ARGS;
+#endif
+args->size = size;
+
+/* Caching the pre-processed capture list. */
+if (cb->last_data != NULL)
+ cb->last_data->next = &args->header;
+else
+ cb->first_data = &args->header;
+
+cb->last_data = &args->header;
+
+/* Create the capture list size. */
+
+captures = (uint16_t*)(args + 1);
+
+while (TRUE)
+ {
+ ++pptr;
+
+ switch (META_CODE(*pptr))
+ {
+ case META_OFFSET:
+ SKIPOFFSET(pptr);
+ continue;
+
+ case META_CAPTURE_NAME:
+ ng = cb->named_groups + *(++pptr);
+ PCRE2_ASSERT((ng->hash_dup & NAMED_GROUP_IS_DUPNAME) != 0);
+ *captures++ = (uint16_t)(ng->number);
+
+ name = ng->name;
+
+ while (++ng < end)
+ if (ng->name == name) *captures++ = (uint16_t)(ng->number);
+ continue;
+
+ case META_CAPTURE_NUMBER:
+ *captures++ = *(++pptr);
+ continue;
+
+ default:
+ break;
+ }
+
+ break;
+ }
+
+PCRE2_ASSERT(size == (size_t)(captures - (uint16_t*)(args + 1)));
+args->skip_size = (size_t)(pptr - pptr_start) - 1;
+
+if (size == 1) return TRUE;
+
+/* Sort captures. */
+
+captures = (uint16_t*)(args + 1);
+i = (size >> 1) - 1;
+while (TRUE)
+ {
+ do_heapify_u16(captures, size, i);
+ if (i == 0) break;
+ i--;
+ }
+
+for (i = size - 1; i > 0; i--)
+ {
+ tmp = captures[0];
+ captures[0] = captures[i];
+ captures[i] = tmp;
+
+ do_heapify_u16(captures, i, 0);
+ }
+
+/* Remove duplicates. */
+
+captures_end = captures + size;
+tmp = *captures++;
+current = captures;
+
+while (current < captures_end)
+ {
+ if (*current != tmp)
+ {
+ tmp = *current;
+ *captures++ = tmp;
+ }
+
+ current++;
+ }
+
+args->size = (size_t)(captures - (uint16_t*)(args + 1));
+return TRUE;
+}
+
+/* End of pcre2_compile_cgroup.c */
diff --git a/src/pcre2_compile_class.c b/src/pcre2_compile_class.c
index 6a73bb9..9a1fc02 100644
--- a/src/pcre2_compile_class.c
+++ b/src/pcre2_compile_class.c
@@ -38,12 +38,11 @@ POSSIBILITY OF SUCH DAMAGE.
-----------------------------------------------------------------------------
*/
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
#include "pcre2_compile.h"
+
+
typedef struct {
/* Option bits for eclass. */
uint32_t options;
@@ -66,7 +65,7 @@ b) none of the cases here:
#define CLASS_END_CASES(meta) \
default: \
PCRE2_ASSERT((meta) <= META_END); \
- /* Fall through */ \
+ PCRE2_FALLTHROUGH /* Fall through */ \
case META_CLASS: \
case META_CLASS_NOT: \
case META_CLASS_EMPTY: \
@@ -525,6 +524,9 @@ if (xoptions & PCRE2_EXTRA_CASELESS_RESTRICT)
if (xoptions & PCRE2_EXTRA_TURKISH_CASING)
class_options |= PARSE_CLASS_TURKISH_UTF;
+#else
+(void)options; /* Avoid compiler warning. */
+(void)xoptions; /* Avoid compiler warning. */
#endif
/* Compute required space for the range. */
@@ -543,7 +545,10 @@ cranges = cb->cx->memctl.malloc(
if (cranges == NULL) return NULL;
-cranges->next = NULL;
+cranges->header.next = NULL;
+#ifdef PCRE2_DEBUG
+cranges->header.type = CDATA_CRANGE;
+#endif
cranges->range_list_size = (uint16_t)range_list_size;
cranges->char_lists_types = 0;
cranges->char_lists_size = 0;
@@ -905,6 +910,10 @@ uint8_t *classbits = cb->classbits.classbits;
uint32_t c, byte_start, byte_end;
uint32_t classbits_end = (end <= 0xff ? end : 0xff);
+#ifndef SUPPORT_UNICODE
+(void)xoptions; /* Avoid compiler warning. */
+#endif
+
/* If caseless matching is required, scan the range and process alternate
cases. In Unicode, there are 8-bit characters that have alternate cases that
are greater than 255 and vice-versa (though these may be ignored if caseless
@@ -1080,6 +1089,10 @@ BOOL utf = FALSE;
uint32_t xclass_props;
PCRE2_UCHAR *class_uchardata;
class_ranges* cranges;
+#else
+(void)has_bitmap; /* Avoid compiler warning. */
+(void)errorcodeptr; /* Avoid compiler warning. */
+(void)lengthptr; /* Avoid compiler warning. */
#endif
/* If an XClass contains a negative special such as \S, we need to flip the
@@ -1112,19 +1125,19 @@ if (utf)
}
/* Caching the pre-processed character ranges. */
- if (cb->next_cranges != NULL)
- cb->next_cranges->next = cranges;
+ if (cb->last_data != NULL)
+ cb->last_data->next = &cranges->header;
else
- cb->cranges = cranges;
+ cb->first_data = &cranges->header;
- cb->next_cranges = cranges;
+ cb->last_data = &cranges->header;
}
else
{
/* Reuse the pre-processed character ranges. */
- cranges = cb->cranges;
- PCRE2_ASSERT(cranges != NULL);
- cb->cranges = cranges->next;
+ cranges = (class_ranges*)cb->first_data;
+ PCRE2_ASSERT(cranges != NULL && cranges->header.type == CDATA_CRANGE);
+ cb->first_data = cranges->header.next;
}
if (cranges->range_list_size > 0)
@@ -1270,8 +1283,23 @@ while (TRUE)
value of 1 removes vertical space and 2 removes underscore. */
if (tabopt < 0) tabopt = -tabopt;
+#ifdef EBCDIC
+ {
+ uint8_t posix_vertical[4] = { CHAR_LF, CHAR_VT, CHAR_FF, CHAR_CR };
+ uint8_t posix_underscore = CHAR_UNDERSCORE;
+ uint8_t *chars = NULL;
+ int n = 0;
+
+ if (tabopt == 1) { chars = posix_vertical; n = 4; }
+ else if (tabopt == 2) { chars = &posix_underscore; n = 1; }
+
+ for (; n > 0; ++chars, --n)
+ pbits.classbits[*chars/8] &= ~(1u << (*chars&7));
+ }
+#else
if (tabopt == 1) pbits.classbits[1] &= ~0x3c;
- else if (tabopt == 2) pbits.classbits[11] &= 0x7f;
+ else if (tabopt == 2) pbits.classbits[11] &= 0x7f;
+#endif
/* Add the POSIX table or its complement into the main table that is
being built and we are done. */
@@ -2079,9 +2107,11 @@ switch (op)
lhs_op_info->bits.classwords[i] ^= rhs_op_info->bits.classwords[i];
break;
+ /* LCOV_EXCL_START */
default:
PCRE2_DEBUG_UNREACHABLE();
break;
+ /* LCOV_EXCL_STOP */
}
}
@@ -2141,7 +2171,7 @@ switch (meta)
}
ptr++;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
default:
/* Scan forward characters, ranges, and properties.
@@ -2158,11 +2188,13 @@ switch (meta)
/* We must have a 100% guarantee that ptr increases when
compile_class_operand() returns, even on Release builds, so that we can
statically prove our loops terminate. */
+ /* LCOV_EXCL_START */
if (ptr <= prev_ptr)
{
PCRE2_DEBUG_UNREACHABLE();
return FALSE;
}
+ /* LCOV_EXCL_STOP */
/* If we fell through above, consume the closing ']'. */
if (meta == META_CLASS || meta == META_CLASS_NOT)
diff --git a/src/pcre2_config.c b/src/pcre2_config.c
index 031981b..045f6e7 100644
--- a/src/pcre2_config.c
+++ b/src/pcre2_config.c
@@ -38,17 +38,10 @@ POSSIBILITY OF SUCH DAMAGE.
-----------------------------------------------------------------------------
*/
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-/* Save the configured link size, which is in bytes. In 16-bit and 32-bit modes
-its value gets changed by pcre2_intmodedep.h (included by pcre2_internal.h) to
-be in code units. */
+#include "pcre2_internal.h"
-static int configured_link_size = LINK_SIZE;
-#include "pcre2_internal.h"
/* These macros are the standard way of turning unquoted text into C strings.
They allow macros like PCRE2_MAJOR to be defined without quotes, which is
@@ -79,7 +72,7 @@ pcre2_config(uint32_t what, void *where)
{
if (where == NULL) /* Requests a length */
{
- switch(what)
+ switch (what)
{
default:
return PCRE2_ERROR_BADOPTION;
@@ -87,6 +80,7 @@ if (where == NULL) /* Requests a length */
case PCRE2_CONFIG_BSR:
case PCRE2_CONFIG_COMPILED_WIDTHS:
case PCRE2_CONFIG_DEPTHLIMIT:
+ case PCRE2_CONFIG_EFFECTIVE_LINKSIZE:
case PCRE2_CONFIG_HEAPLIMIT:
case PCRE2_CONFIG_JIT:
case PCRE2_CONFIG_LINKSIZE:
@@ -124,13 +118,13 @@ switch (what)
case PCRE2_CONFIG_COMPILED_WIDTHS:
*((uint32_t *)where) = 0
#ifdef SUPPORT_PCRE2_8
- + 1
+ + (1 << 0)
#endif
#ifdef SUPPORT_PCRE2_16
- + 2
+ + (1 << 1)
#endif
#ifdef SUPPORT_PCRE2_32
- + 4
+ + (1 << 2)
#endif
;
break;
@@ -139,6 +133,10 @@ switch (what)
*((uint32_t *)where) = MATCH_LIMIT_DEPTH;
break;
+ case PCRE2_CONFIG_EFFECTIVE_LINKSIZE:
+ *((uint32_t *)where) = LINK_SIZE * sizeof(PCRE2_UCHAR);
+ break;
+
case PCRE2_CONFIG_HEAPLIMIT:
*((uint32_t *)where) = HEAP_LIMIT;
break;
@@ -163,7 +161,7 @@ switch (what)
#endif
case PCRE2_CONFIG_LINKSIZE:
- *((uint32_t *)where) = (uint32_t)configured_link_size;
+ *((uint32_t *)where) = (uint32_t)CONFIGURED_LINK_SIZE;
break;
case PCRE2_CONFIG_MATCHLIMIT:
@@ -206,8 +204,7 @@ switch (what)
#endif
return (int)(1 + ((where == NULL)?
strlen(v) : PRIV(strcpy_c8)((PCRE2_UCHAR *)where, v)));
- }
- break;
+ }
case PCRE2_CONFIG_UNICODE:
#if defined SUPPORT_UNICODE
@@ -244,6 +241,7 @@ switch (what)
return (int)(1 + ((where == NULL)?
strlen(v) : PRIV(strcpy_c8)((PCRE2_UCHAR *)where, v)));
}
+
}
return 0;
diff --git a/src/pcre2_context.c b/src/pcre2_context.c
index 2345145..6246683 100644
--- a/src/pcre2_context.c
+++ b/src/pcre2_context.c
@@ -39,10 +39,6 @@ POSSIBILITY OF SUCH DAMAGE.
*/
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-
#include "pcre2_internal.h"
@@ -506,10 +502,7 @@ return 0;
}
/* These functions became obsolete at release 10.30. The first is kept as a
-synonym for backwards compatibility. The second now does nothing. Exclude both
-from coverage reports. */
-
-/* LCOV_EXCL_START */
+synonym for backwards compatibility. The second now does nothing. */
PCRE2_EXP_DEFN int PCRE2_CALL_CONVENTION
pcre2_set_recursion_limit(pcre2_match_context *mcontext, uint32_t limit)
@@ -529,8 +522,6 @@ pcre2_set_recursion_memory_management(pcre2_match_context *mcontext,
return 0;
}
-/* LCOV_EXCL_STOP */
-
/* ------------ Convert context ------------ */
@@ -543,10 +534,20 @@ ccontext->glob_separator = separator;
return 0;
}
+static const char *globpunct =
+ STR_EXCLAMATION_MARK STR_QUOTATION_MARK STR_NUMBER_SIGN STR_DOLLAR_SIGN
+ STR_PERCENT_SIGN STR_AMPERSAND STR_APOSTROPHE STR_LEFT_PARENTHESIS
+ STR_RIGHT_PARENTHESIS STR_ASTERISK STR_PLUS STR_COMMA STR_MINUS STR_DOT
+ STR_SLASH STR_COLON STR_SEMICOLON STR_LESS_THAN_SIGN STR_EQUALS_SIGN
+ STR_GREATER_THAN_SIGN STR_QUESTION_MARK STR_COMMERCIAL_AT
+ STR_LEFT_SQUARE_BRACKET STR_BACKSLASH STR_RIGHT_SQUARE_BRACKET
+ STR_CIRCUMFLEX_ACCENT STR_UNDERSCORE STR_GRAVE_ACCENT STR_LEFT_CURLY_BRACKET
+ STR_VERTICAL_LINE STR_RIGHT_CURLY_BRACKET STR_TILDE;
+
PCRE2_EXP_DEFN int PCRE2_CALL_CONVENTION
pcre2_set_glob_escape(pcre2_convert_context *ccontext, uint32_t escape)
{
-if (escape > 255 || (escape != 0 && !ispunct(escape)))
+if (escape > 255 || (escape != 0 && strchr(globpunct, escape) == NULL))
return PCRE2_ERROR_BADDATA;
ccontext->glob_escape = escape;
return 0;
diff --git a/src/pcre2_convert.c b/src/pcre2_convert.c
index d2b238c..ad7312a 100644
--- a/src/pcre2_convert.c
+++ b/src/pcre2_convert.c
@@ -39,12 +39,10 @@ POSSIBILITY OF SUCH DAMAGE.
*/
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-
#include "pcre2_internal.h"
+
+
#define TYPE_OPTIONS (PCRE2_CONVERT_GLOB| \
PCRE2_CONVERT_POSIX_BASIC|PCRE2_CONVERT_POSIX_EXTENDED)
@@ -81,6 +79,16 @@ enum { POSIX_START_REGEX, POSIX_ANCHORED, POSIX_NOT_BRACKET,
} \
}
+/* Macro to check for lowercase characters. */
+
+#ifdef EBCDIC
+#define ISLOWER(c) (((c) >= CHAR_a && (c) <= CHAR_i) || \
+ ((c) >= CHAR_j && (c) <= CHAR_r) || \
+ ((c) >= CHAR_s && (c) <= CHAR_z))
+#else
+#define ISLOWER(c) ((c) >= CHAR_a && (c) <= CHAR_z)
+#endif
+
/* Literals that must be escaped: \ ? * + | . ^ $ { } [ ] ( ) */
static const char *pcre2_escaped_literals =
@@ -97,6 +105,24 @@ static const char *posix_meta_escapes =
STR_LEFT_CURLY_BRACKET STR_RIGHT_CURLY_BRACKET
STR_1 STR_2 STR_3 STR_4 STR_5 STR_6 STR_7 STR_8 STR_9;
+/* Recognized POSIX classes, colon-separated. */
+
+static const char *posix_classes =
+ STR_a STR_l STR_p STR_h STR_a STR_COLON
+ STR_l STR_o STR_w STR_e STR_r STR_COLON
+ STR_u STR_p STR_p STR_e STR_r STR_COLON
+ STR_a STR_l STR_n STR_u STR_m STR_COLON
+ STR_a STR_s STR_c STR_i STR_i STR_COLON
+ STR_b STR_l STR_a STR_n STR_k STR_COLON
+ STR_c STR_n STR_t STR_r STR_l STR_COLON
+ STR_d STR_i STR_g STR_i STR_t STR_COLON
+ STR_g STR_r STR_a STR_p STR_h STR_COLON
+ STR_p STR_r STR_i STR_n STR_t STR_COLON
+ STR_p STR_u STR_n STR_c STR_t STR_COLON
+ STR_s STR_p STR_a STR_c STR_e STR_COLON
+ STR_w STR_o STR_r STR_d STR_COLON
+ STR_x STR_d STR_i STR_g STR_i STR_t STR_COLON;
+
/*************************************************
@@ -190,7 +216,7 @@ while (plength > 0)
switch (posix_state)
{
case POSIX_CLASS_STARTED:
- if (c <= 127 && islower(c)) break; /* Remain in started state */
+ if (ISLOWER(c)) break; /* Remain in started state */
posix_state = POSIX_CLASS_NOT_STARTED;
if (c == CHAR_COLON && plength > 0 &&
*posix == CHAR_RIGHT_SQUARE_BRACKET)
@@ -200,7 +226,7 @@ while (plength > 0)
posix++;
continue; /* With next character after :] */
}
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case POSIX_CLASS_NOT_STARTED:
if (c == CHAR_LEFT_SQUARE_BRACKET)
@@ -277,9 +303,9 @@ while (plength > 0)
if (plength == 0) return PCRE2_ERROR_END_BACKSLASH;
if (extended) nextisliteral = TRUE; else
{
- if (*posix < 127 && strchr(posix_meta_escapes, *posix) != NULL)
+ if (*posix < 255 && strchr(posix_meta_escapes, *posix) != NULL)
{
- if (isdigit(*posix)) PUTCHARS(STR_BACKSLASH);
+ if (*posix >= CHAR_0 && *posix <= CHAR_9) PUTCHARS(STR_BACKSLASH);
if (p + 1 > endp) return PCRE2_ERROR_NOMEMORY;
lastspecial = *p++ = *posix++;
plength--;
@@ -295,7 +321,7 @@ while (plength > 0)
case CHAR_LEFT_PARENTHESIS:
bracount++;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case CHAR_QUESTION_MARK:
case CHAR_PLUS:
@@ -303,7 +329,7 @@ while (plength > 0)
case CHAR_RIGHT_CURLY_BRACKET:
case CHAR_VERTICAL_LINE:
if (!extended) goto ESCAPE_LITERAL;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case CHAR_DOT:
case CHAR_DOLLAR_SIGN:
@@ -332,10 +358,10 @@ while (plength > 0)
posix_state = POSIX_ANCHORED;
goto COPY_SPECIAL;
}
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
default:
- if (c < 128 && strchr(pcre2_escaped_literals, c) != NULL)
+ if (c < 255 && strchr(pcre2_escaped_literals, c) != NULL)
{
ESCAPE_LITERAL:
PUTCHARS(STR_BACKSLASH);
@@ -474,8 +500,6 @@ static int
convert_glob_parse_class(PCRE2_SPTR *from, PCRE2_SPTR pattern_end,
pcre2_output_context *out)
{
-static const char *posix_classes = "alnum:alpha:ascii:blank:cntrl:digit:"
- "graph:lower:print:punct:space:upper:word:xdigit:";
PCRE2_SPTR start = *from + 1;
PCRE2_SPTR pattern = start;
const char *class_ptr;
@@ -500,7 +524,7 @@ class_index = 1;
while (TRUE)
{
- if (*class_ptr == CHAR_NUL) return 0;
+ if (*class_ptr == 0) return 0;
pattern = start;
@@ -539,31 +563,61 @@ Returns: !0 => character is found in the class
static BOOL
convert_glob_char_in_class(int class_index, PCRE2_UCHAR c)
{
+const uint8_t *cbits = PRIV(default_tables) + cbits_offset;
+int cbit;
+
#if PCRE2_CODE_UNIT_WIDTH != 8
if (c > 0xff)
{
- /* ctype functions are not sane for c > 0xff */
- return 0;
+ /* Can't access the character tables for c > 0xff */
+ return FALSE;
}
#endif
+/* See posix_class_maps. This is a small local clone of that.
+Note that we don't know exactly what character tables will be used at
+match time, but, for the purposes of pattern conversion, it should be
+sufficient to use PCRE2's built-in default tables. */
+
switch (class_index)
{
- case 1: return isalnum(c);
- case 2: return isalpha(c);
- case 3: return 1;
- case 4: return c == CHAR_HT || c == CHAR_SPACE;
- case 5: return iscntrl(c);
- case 6: return isdigit(c);
- case 7: return isgraph(c);
- case 8: return islower(c);
- case 9: return isprint(c);
- case 10: return ispunct(c);
- case 11: return isspace(c);
- case 12: return isupper(c);
- case 13: return isalnum(c) || c == CHAR_UNDERSCORE;
- default: return isxdigit(c);
+ case 1: /* alpha */
+ if (c == CHAR_UNDERSCORE) return FALSE;
+ if (((cbits + cbit_digit)[c/8] & (1u << (c&7))) != 0) return FALSE;
+ cbit = cbit_word;
+ break;
+
+ case 2: cbit = cbit_lower; break; /* lower */
+ case 3: cbit = cbit_upper; break; /* upper */
+
+ case 4: /* alnum */
+ if (c == CHAR_UNDERSCORE) return FALSE;
+ cbit = cbit_word;
+ break;
+
+ case 5: /* ascii */
+ if (((cbits + cbit_cntrl)[c/8] & (1u << (c&7))) != 0) return TRUE;
+ cbit = cbit_print;
+ break;
+
+ case 6: /* blank */
+ if (c == CHAR_LF || c == CHAR_VT || c == CHAR_FF || c == CHAR_CR)
+ return FALSE;
+ cbit = cbit_space;
+ break;
+
+ case 7: cbit = cbit_cntrl; break; /* cntrl */
+ case 8: cbit = cbit_digit; break; /* digit */
+ case 9: cbit = cbit_graph; break; /* graph */
+ case 10: cbit = cbit_print; break; /* print */
+ case 11: cbit = cbit_punct; break; /* punct */
+ case 12: cbit = cbit_space; break; /* space */
+ case 13: cbit = cbit_word; break; /* word */
+ case 14: cbit = cbit_xdigit; break; /* xdigit */
+ default: return FALSE;
}
+
+return ((cbits + cbit)[c/8] & (1u << (c&7))) != 0;
}
/* Parse a range of characters.
@@ -1005,7 +1059,7 @@ while (pattern < pattern_end)
c = *pattern++;
}
- if (c < 128 && strchr(pcre2_escaped_literals, c) != NULL)
+ if (c < 255 && strchr(pcre2_escaped_literals, c) != NULL)
convert_glob_write(&out, CHAR_BACKSLASH);
convert_glob_write(&out, c);
@@ -1065,13 +1119,21 @@ pcre2_pattern_convert(PCRE2_SPTR pattern, PCRE2_SIZE plength, uint32_t options,
pcre2_convert_context *ccontext)
{
int rc;
+PCRE2_UCHAR null_str[1] = { 0xcd };
PCRE2_UCHAR dummy_buffer[DUMMY_BUFFER_SIZE];
PCRE2_UCHAR *use_buffer = dummy_buffer;
PCRE2_SIZE use_length = DUMMY_BUFFER_SIZE;
BOOL utf = (options & PCRE2_CONVERT_UTF) != 0;
uint32_t pattype = options & TYPE_OPTIONS;
-if (pattern == NULL || bufflenptr == NULL) return PCRE2_ERROR_NULL;
+if (pattern == NULL && plength == 0)
+ pattern = null_str;
+
+if (pattern == NULL || bufflenptr == NULL)
+ {
+ if (bufflenptr != NULL) *bufflenptr = 0; /* Error offset */
+ return PCRE2_ERROR_NULL;
+ }
if ((options & ~ALL_OPTIONS) != 0 || /* Undefined bit set */
(pattype & (~pattype+1)) != pattype || /* More than one type set */
@@ -1136,8 +1198,13 @@ for (int i = 0; i < 2; i++)
bufflenptr, dummyrun, ccontext);
break;
+ /* We have already validated pattype. */
+ /* LCOV_EXCL_START */
default:
- goto EXIT;
+ PCRE2_DEBUG_UNREACHABLE();
+ *bufflenptr = 0; /* Error offset */
+ return PCRE2_ERROR_INTERNAL;
+ /* LCOV_EXCL_STOP */
}
if (rc != 0 || /* Error */
@@ -1150,20 +1217,23 @@ for (int i = 0; i < 2; i++)
allocated = PRIV(memctl_malloc)(sizeof(pcre2_memctl) +
(*bufflenptr + 1)*PCRE2_CODE_UNIT_WIDTH, (pcre2_memctl *)ccontext);
- if (allocated == NULL) return PCRE2_ERROR_NOMEMORY;
+ if (allocated == NULL)
+ {
+ *bufflenptr = 0; /* Error offset */
+ return PCRE2_ERROR_NOMEMORY;
+ }
*buffptr = (PCRE2_UCHAR *)(((char *)allocated) + sizeof(pcre2_memctl));
use_buffer = *buffptr;
use_length = *bufflenptr + 1;
}
-/* Something went terribly wrong. Trigger an assert and return an error */
+/* Running the loop above ought to have succeeded the second time. */
+/* LCOV_EXCL_START */
PCRE2_DEBUG_UNREACHABLE();
-
-EXIT:
-
*bufflenptr = 0; /* Error offset */
return PCRE2_ERROR_INTERNAL;
+/* LCOV_EXCL_STOP */
}
diff --git a/src/pcre2_dfa_match.c b/src/pcre2_dfa_match.c
index ebf31d2..f507acf 100644
--- a/src/pcre2_dfa_match.c
+++ b/src/pcre2_dfa_match.c
@@ -72,16 +72,14 @@ Overall, I concluded that the gains in some cases did not outweigh the losses
in others, so I abandoned this code. */
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
+#include "pcre2_internal.h"
+
+
#define NLBLOCK mb /* Block containing newline information */
#define PSSTART start_subject /* Field containing processed string start */
#define PSEND end_subject /* Field containing processed string end */
-#include "pcre2_internal.h"
-
#define PUBLIC_DFA_MATCH_OPTIONS \
(PCRE2_ANCHORED|PCRE2_ENDANCHORED|PCRE2_NOTBOL|PCRE2_NOTEOL|PCRE2_NOTEMPTY| \
PCRE2_NOTEMPTY_ATSTART|PCRE2_NO_UTF_CHECK|PCRE2_PARTIAL_HARD| \
@@ -2330,7 +2328,7 @@ for (;;)
case 0x2029:
#endif /* Not EBCDIC */
if (mb->bsr_convention == PCRE2_BSR_ANYCRLF) break;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case CHAR_LF:
ADD_NEW(state_offset + 1, 0);
@@ -2442,7 +2440,7 @@ for (;;)
caseless = TRUE;
codevalue -= OP_STARI - OP_STAR;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_PLUS:
case OP_MINPLUS:
case OP_POSPLUS:
@@ -2486,7 +2484,7 @@ for (;;)
case OP_NOTPOSQUERYI:
caseless = TRUE;
codevalue -= OP_STARI - OP_STAR;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_QUERY:
case OP_MINQUERY:
case OP_POSQUERY:
@@ -2527,7 +2525,7 @@ for (;;)
case OP_NOTPOSSTARI:
caseless = TRUE;
codevalue -= OP_STARI - OP_STAR;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_STAR:
case OP_MINSTAR:
case OP_POSSTAR:
@@ -2564,7 +2562,7 @@ for (;;)
case OP_NOTEXACTI:
caseless = TRUE;
codevalue -= OP_STARI - OP_STAR;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_EXACT:
case OP_NOTEXACT:
count = current_state->count; /* Number already matched */
@@ -2599,7 +2597,7 @@ for (;;)
case OP_NOTPOSUPTOI:
caseless = TRUE;
codevalue -= OP_STARI - OP_STAR;
- /* Fall through */
+ PCRE2_FALLTHROUGH /* Fall through */
case OP_UPTO:
case OP_MINUPTO:
case OP_POSUPTO:
@@ -2941,6 +2939,9 @@ for (;;)
uint32_t recno = (callpat == mb->start_code)? 0 :
GET2(callpat, 1 + LINK_SIZE);
+ /* Argument list has not been supported yet. */
+ if (code[1 + LINK_SIZE] == OP_CREF) return PCRE2_ERROR_DFA_UITEM;
+
if (rws->free < RWS_RSIZE + RWS_OVEC_RSIZE)
{
rc = more_workspace(&rws, RWS_OVEC_RSIZE, mb);
@@ -3341,10 +3342,12 @@ pcre2_dfa_match(const pcre2_code *code, PCRE2_SPTR subject, PCRE2_SIZE length,
pcre2_match_context *mcontext, int *workspace, PCRE2_SIZE wscount)
{
int rc;
-int was_zero_terminated = 0;
const pcre2_real_code *re = (const pcre2_real_code *)code;
+uint32_t original_options = options;
+PCRE2_UCHAR null_str[1] = { 0xcd };
+PCRE2_SPTR original_subject = subject;
PCRE2_SPTR start_match;
PCRE2_SPTR end_subject;
PCRE2_SPTR bumpalong_limit;
@@ -3386,44 +3389,46 @@ rws->free = RWS_BASE_SIZE - RWS_ANCHOR_SIZE;
/* Recognize NULL, length 0 as an empty string. */
-if (subject == NULL && length == 0) subject = (PCRE2_SPTR)"";
+if (subject == NULL && length == 0) subject = null_str;
/* Plausibility checks */
-if ((options & ~PUBLIC_DFA_MATCH_OPTIONS) != 0) return PCRE2_ERROR_BADOPTION;
-if (re == NULL || subject == NULL || workspace == NULL || match_data == NULL)
- return PCRE2_ERROR_NULL;
+if (match_data == NULL) return PCRE2_ERROR_NULL;
+if (re == NULL || subject == NULL || workspace == NULL)
+ { rc = PCRE2_ERROR_NULL; goto EXIT; }
+if ((options & ~PUBLIC_DFA_MATCH_OPTIONS) != 0)
+ { rc = PCRE2_ERROR_BADOPTION; goto EXIT; }
if (length == PCRE2_ZERO_TERMINATED)
{
length = PRIV(strlen)(subject);
- was_zero_terminated = 1;
}
-if (wscount < 20) return PCRE2_ERROR_DFA_WSSIZE;
-if (start_offset > length) return PCRE2_ERROR_BADOFFSET;
+if (wscount < 20) { rc = PCRE2_ERROR_DFA_WSSIZE; goto EXIT; }
+if (start_offset > length) { rc = PCRE2_ERROR_BADOFFSET; goto EXIT; }
/* Partial matching and PCRE2_ENDANCHORED are currently not allowed at the same
time. */
if ((options & (PCRE2_PARTIAL_HARD|PCRE2_PARTIAL_SOFT)) != 0 &&
((re->overall_options | options) & PCRE2_ENDANCHORED) != 0)
- return PCRE2_ERROR_BADOPTION;
+ { rc = PCRE2_ERROR_BADOPTION; goto EXIT; }
/* Invalid UTF support is not available for DFA matching. */
if ((re->overall_options & PCRE2_MATCH_INVALID_UTF) != 0)
- return PCRE2_ERROR_DFA_UINVALID_UTF;
+ { rc = PCRE2_ERROR_DFA_UINVALID_UTF; goto EXIT; }
/* Check that the first field in the block is the magic number. If it is not,
return with PCRE2_ERROR_BADMAGIC. */
-if (re->magic_number != MAGIC_NUMBER) return PCRE2_ERROR_BADMAGIC;
+if (re->magic_number != MAGIC_NUMBER)
+ { rc = PCRE2_ERROR_BADMAGIC; goto EXIT; }
/* Check the code unit width. */
if ((re->flags & PCRE2_MODE_MASK) != PCRE2_CODE_UNIT_WIDTH/8)
- return PCRE2_ERROR_BADMODE;
+ { rc = PCRE2_ERROR_BADMODE; goto EXIT; }
/* PCRE2_NOTEMPTY and PCRE2_NOTEMPTY_ATSTART are match-time flags in the
options variable for this function. Users of PCRE2 who are not calling the
@@ -3449,8 +3454,8 @@ of the workspace. */
if ((options & PCRE2_DFA_RESTART) != 0)
{
if ((workspace[0] & (-2)) != 0 || workspace[1] < 1 ||
- workspace[1] > (int)((wscount - 2)/INTS_PER_STATEBLOCK))
- return PCRE2_ERROR_DFA_BADRESTART;
+ workspace[1] > (int)((wscount - 2)/INTS_PER_STATEBLOCK))
+ { rc = PCRE2_ERROR_DFA_BADRESTART; goto EXIT; }
}
/* Set some local values */
@@ -3498,7 +3503,7 @@ else
if (mcontext->offset_limit != PCRE2_UNSET)
{
if ((re->overall_options & PCRE2_USE_OFFSET_LIMIT) == 0)
- return PCRE2_ERROR_BADOFFSETLIMIT;
+ { rc = PCRE2_ERROR_BADOFFSETLIMIT; goto EXIT; }
bumpalong_limit = subject + mcontext->offset_limit;
}
mb->callout = mcontext->callout;
@@ -3565,9 +3570,12 @@ switch(re->newline_convention)
mb->nltype = NLTYPE_ANYCRLF;
break;
+ /* LCOV_EXCL_START */
default:
PCRE2_DEBUG_UNREACHABLE();
- return PCRE2_ERROR_INTERNAL;
+ rc = PCRE2_ERROR_INTERNAL;
+ goto EXIT;
+ /* LCOV_EXCL_STOP */
}
/* Check a UTF string for validity if required. For 8-bit and 16-bit strings,
@@ -3588,7 +3596,7 @@ if (utf && (options & PCRE2_NO_UTF_CHECK) == 0)
#if PCRE2_CODE_UNIT_WIDTH != 32
unsigned int i;
if (start_match < end_subject && NOT_FIRSTCU(*start_match))
- return PCRE2_ERROR_BADUTFOFFSET;
+ { rc = PCRE2_ERROR_BADUTFOFFSET; goto EXIT; }
for (i = re->max_lookbehind; i > 0 && check_subject > subject; i--)
{
check_subject--;
@@ -3609,12 +3617,12 @@ if (utf && (options & PCRE2_NO_UTF_CHECK) == 0)
/* Validate the relevant portion of the subject. After an error, adjust the
offset to be an absolute offset in the whole string. */
- match_data->rc = PRIV(valid_utf)(check_subject,
+ rc = PRIV(valid_utf)(check_subject,
length - (PCRE2_SIZE)(check_subject - subject), &(match_data->startchar));
- if (match_data->rc != 0)
+ if (rc != 0)
{
match_data->startchar += (PCRE2_SIZE)(check_subject - subject);
- return match_data->rc;
+ goto EXIT;
}
}
#endif /* SUPPORT_UNICODE */
@@ -3678,9 +3686,10 @@ if ((match_data->flags & PCRE2_MD_COPIED_SUBJECT) != 0)
/* Fill in fields that are always returned in the match data. */
match_data->code = re;
-match_data->subject = NULL; /* Default for no match */
+match_data->subject = NULL; /* Default for match error */
match_data->mark = NULL;
match_data->matchedby = PCRE2_MATCHEDBY_DFA_INTERPRETER;
+match_data->options = original_options;
/* Call the main matching function, looping for a non-anchored regex after a
failed match. If not restarting, perform certain optimizations at the start of
@@ -4032,29 +4041,40 @@ for (;;)
if (rc != PCRE2_ERROR_NOMATCH || anchored)
{
+ if (rc == PCRE2_ERROR_NOMATCH) goto NOMATCH_EXIT;
+
if (rc == PCRE2_ERROR_PARTIAL && match_data->oveccount > 0)
{
match_data->ovector[0] = (PCRE2_SIZE)(start_match - subject);
match_data->ovector[1] = (PCRE2_SIZE)(end_subject - subject);
}
- match_data->subject_length = length;
- match_data->leftchar = (PCRE2_SIZE)(mb->start_used_ptr - subject);
- match_data->rightchar = (PCRE2_SIZE)(mb->last_used_ptr - subject);
- match_data->startchar = (PCRE2_SIZE)(start_match - subject);
- match_data->rc = rc;
- if (rc >= 0 &&(options & PCRE2_COPY_MATCHED_SUBJECT) != 0)
+ if (rc >= 0 || rc == PCRE2_ERROR_PARTIAL)
+ {
+ match_data->subject_length = length;
+ match_data->start_offset = start_offset;
+ match_data->leftchar = (PCRE2_SIZE)(mb->start_used_ptr - subject);
+ match_data->rightchar = (PCRE2_SIZE)(mb->last_used_ptr - subject);
+ match_data->startchar = (PCRE2_SIZE)(start_match - subject);
+ }
+
+ if (rc >= 0 && (options & PCRE2_COPY_MATCHED_SUBJECT) != 0)
{
- length = CU2BYTES(length + was_zero_terminated);
- match_data->subject = match_data->memctl.malloc(length,
- match_data->memctl.memory_data);
- if (match_data->subject == NULL) return PCRE2_ERROR_NOMEMORY;
- memcpy((void *)match_data->subject, subject, length);
+ if (length != 0)
+ {
+ match_data->subject = match_data->memctl.malloc(CU2BYTES(length),
+ match_data->memctl.memory_data);
+ if (match_data->subject == NULL)
+ { rc = PCRE2_ERROR_NOMEMORY; goto EXIT; }
+ memcpy((void *)match_data->subject, subject, CU2BYTES(length));
+ }
+ else
+ match_data->subject = NULL;
match_data->flags |= PCRE2_MD_COPIED_SUBJECT;
}
- else
+ else if (rc >= 0 || rc == PCRE2_ERROR_PARTIAL)
{
- if (rc >= 0 || rc == PCRE2_ERROR_PARTIAL) match_data->subject = subject;
+ match_data->subject = original_subject;
}
goto EXIT;
}
@@ -4088,6 +4108,9 @@ for (;;)
} /* "Bumpalong" loop */
NOMATCH_EXIT:
+match_data->subject = original_subject;
+match_data->subject_length = length;
+match_data->start_offset = start_offset;
rc = PCRE2_ERROR_NOMATCH;
EXIT:
@@ -4098,6 +4121,7 @@ while (rws->next != NULL)
mb->memctl.free(next, mb->memctl.memory_data);
}
+match_data->rc = rc;
return rc;
}
diff --git a/src/pcre2_dftables.c b/src/pcre2_dftables.c
index 0f9aedf..1d19870 100644
--- a/src/pcre2_dftables.c
+++ b/src/pcre2_dftables.c
@@ -47,21 +47,23 @@ option can be used to select the current locale from the LC_ALL environment
variable. By default, the tables are written in source form, but if -b is
given, they are written in binary. */
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
+
#include
#include
#include
#include
-#define PCRE2_DFTABLES /* for pcre2_internal.h, pcre2_maketables.c */
+/* For pcre2_internal.h, pcre2_maketables.c, pcre2_tables.c */
+#define PCRE2_DFTABLES
+/* For pcre2_tables.c */
+#define PRIV(name) name
#define PCRE2_CODE_UNIT_WIDTH 0 /* Must be set, but not relevant here */
#include "pcre2_internal.h"
#include "pcre2_maketables.c"
+#include "pcre2_tables.c"
static const char *classlist[] =
@@ -70,6 +72,23 @@ static const char *classlist[] =
"word", "graph", "print", "punct", "cntrl"
};
+static int identity(int c) { return c; }
+
+#ifdef EBCDIC
+static int ebcdic_to_unicode(int c)
+{
+if (c < 0 || c > 255) abort();
+
+return ebcdic_1047_to_ascii[c];
+}
+
+static int unicode_to_ebcdic(int c)
+{
+if (c < 0 || c > 255) abort();
+
+return ascii_to_ebcdic_1047[c];
+}
+#endif
/*************************************************
@@ -83,6 +102,9 @@ usage(void)
"Usage: pcre2_dftables [options]